Smart enumerations

This afternoon, my team leader checked with me that there really was no way of telling when the current iteration of a foreach loop is the last one. I confirmed the situation, and immediately thought, "Well, why isn't there a way?" I know that you can't tell without peeking ahead, but surely there's a simple way of doing that in a general purpose fashion...

About 15 minutes later, SmartEnumerable<T> was born, or at least something with the same functionality. It chains whatever enumeration you give it (in the same way as a lot of the LINQ calls do) but adds extra information about whether this is the first and/or last element in the enumeration, and the notional index of the element. An example will probably make this clearer. Here's some example code:

using System;
using System.Collections.Generic;

using MiscUtil.Collections;

class Example
{
    static void Main(string[] args)
    {
        List<string> list = new List<string>();
        list.Add("a");
        list.Add("b");
        list.Add("c");
        list.Add("d");
        list.Add("e");
        
        foreach (SmartEnumerable<string>.Entry entry in
                 new SmartEnumerable<string>(list))
        {
            Console.WriteLine ("{0,-7} {1} ({2}) {3}",
                               entry.IsLast  ? "Last ->" : "",
                               entry.Value,
                               entry.Index,
                               entry.IsFirst ? "<- First" : "");
        }
    }
}

The output is as follows:

        a (0) <- First
        b (1)
        c (2)
        d (3)
Last -> e (4)

I'm pretty pleased with that - but annoyed with myself for not thinking of doing it before. I'm pretty shocked that I haven't seen it elsewhere; the code behind it is really straightforward. Anyway, it's now part of my Miscellaneous Utilities library, so feel free to have at it.

Of course, if any of you cunning readers have seen the same thing elsewhere, feel free to indicate just how ignorant I am...

Published Fri, Jul 27 2007 23:50 by skeet
Filed under:

Comments

# re: Smart enumerations

That's really cool.  (I'm also surprised that I haven't seen it elsewhere).

I'd like to see that as a C# language feature.  Maybe if the compiler sees an ICurrentEnumerator implementation then it could allow code to make use of a "current" operator:

foreach (string s in myICurrentEnumeratorImpl)

{

 Console.WriteLine ("{0,-7} {1} ({2}) {3}",

   current.IsLast  ? "Last ->" : "",

   s,

   current.Index,

   current.IsFirst ? "<- First" : "");

}

Friday, July 27, 2007 7:46 PM by Dave

# re: Smart enumerations

That's a neat trick!  But I think the reason why we haven't seen this elsewhere is that in most cases, you can just use a regular "for" loop and check the index instead. :)

Saturday, July 28, 2007 2:40 AM by Chris Nahr

# re: Smart enumerations

You can certainly *often* use a regular for loop instead - although personally I prefer using a foreach where possible. Using the more convoluted "for" form just for the sake of knowing the first-ness/last-ness is a pity. The Index property was added in at the last minute - I wouldn't expect to use that as often :)

Of course, where this is really important is when you *only* have an IEnumerable<T> rather than an IList<T> or similar.

Jon

Saturday, July 28, 2007 3:13 AM by skeet

# re: Smart enumerations

Very useful.  Makes the code much more elegant (in my humble opinion) than the 'traditional' for implementation.

Gold star :)

Sunday, July 29, 2007 12:03 PM by Rohan P

# re: Smart enumerations

Neat, but I think that with C# 3.0 features you might be able to turn this into a method extender (or two) onto the IEnumerator interface.  Would require some thought though, maybe something more along the line of

bool isLast(this IEnumerator<T> list, T item)

{

 return list[list.count-1] == item;

}

Really not sure how the generics would work there actually, and I've yet to write an actual method extension so take that with more then a grain of salt... anyway, might be worth looking into

-mwalts

Monday, July 30, 2007 3:51 PM by mwalts

# re: Smart enumerations

mwalts: That fails on two counts

1) You can't index into an enumerable (or an enumerator), or take its count. There's the Count() extension method of Enumerable, but that requires enumerating through the whole lot if it's performed on anything which isn't an ICollection<T>.

2) You may have the same element multiple times - only the last one should count as being last.

Jon

Monday, July 30, 2007 4:11 PM by skeet

# re: Smart enumerations

Fair enough, I meant it more as a general idea then a specific case, otherwise I would have spent the 5 minutes needed to get myself back up to speed.  I should have remembered the limitations of that interface though which likely preclude anything even related, but it's a Monday, so there you go :p

The idea seems solid for an IList, but yes, that's almost trivial anyway

Frankly, I'm just trying to get my head around the possible uses of method extensions and other C# 3.0 features, since I've really just started my research into them.

-mwalts

Monday, July 30, 2007 4:33 PM by mwalts

# re: Smart enumerations

Yes, I think it'll be a while before extension methods are really well understood and evaluated in terms of the balance between obfuscation and clear expression. (I'm writing about that just now, actually - just taken a break from it for a few minutes!)

I've been finding that the more familiar I become with the C# 3 features (not in terms of knowledge, but in terms of general comfort) the more useful I think they'll be. I used to be quite firmly against implicit typing of local variables apart from when it was needed for anonymous types: I'm a lot more relaxed about it now.

Going back to SmartEnumerable - fortunately even in its current form, it's only 108 lines including XML documentation and the nested Entry class. The significant *code* (within GetEnumerator) is only 17 lines, thanks to iterator blocks.

Extension methods and implicit typing could make the use much nicer too:

foreach (var entry in list.AsSmartEnumerable())

{

   ...

}

Jon

Monday, July 30, 2007 4:53 PM by skeet

# re: Smart enumerations

for loop vs. foreach

Depends on how the collection class is written and you can actually have performance problems either way.

- foreach leaves behind an IEnumerable object that needs to be cleaned up.  So, applications where garbage collection is best avoided foreach is a performance KILLER.  For example, XNA/C# game development.  You have a main game loop firing off maybe 50/second..imagine how much crap you can leave around having multiple nested foreach loops ????

- there is a good MSDN video from back in the days of .net 1.1 or so that describes what is going on

- foreach because of the way it accesses members can be faster if indexers aren't properly implemented.  For example, the SSAS API is screwed up because of this (a couple other reasons too)..but related to this topic foreach is faster than indexers

sqljunkies.com/.../stored_procs_best_practices.aspx

Moral of the story..."convoluted "for" form" is not always that bad and vice-versa as I have learned myself as well over the years :)

Monday, July 30, 2007 11:01 PM by Bart Czernicki

# re: Smart enumerations

IEnumerable doesn't have the ability to tell if it's at the end of the collection, by default, because it may be enumerating something that doesn't have a length.  It may not know until the call to MoveNext whether there is anything following.  See blogs.msdn.com/.../announcement-new-visual-studio-talk-show-podcast.aspx for an example enumerating a something that has no fixed size (and it's end really depends on it's content)

Wednesday, August 01, 2007 1:40 PM by PeterRitchie

# re: Smart enumerations

... but SmartEnumerable<T> *is* useful for fixed-size collections...

Wednesday, August 01, 2007 1:41 PM by PeterRitchie

# re: Smart enumerations

SmartEnumerable<T> doesn't need to only work on fixed-sized collections. It can work on an enumerable that doesn't know its own length beforehand - it's just that MoveNext() will be called before the "current" value is returned. There are a *very* few cases where that would be significant - for instance when the code doing the enumerating indicates that the enumeration should terminate - but so long as it's expected, that's okay.

Wednesday, August 01, 2007 2:07 PM by skeet

# re: Smart enumerations

Jon, yes there are *very* few cases, but those cases need to be supported by IEnumerable; hence it's inability to check--my only point.  I wasn't trying to say SmartEnumerable<T> was useful only for fix-sized collections, just clarifying that I wasn't trying to question the usefulness of it with my previous comment.

Wednesday, August 01, 2007 2:55 PM by PeterRitchie

# re: Smart enumerations

Ah, I see. Just as a clarification of *both* our posts (hopefully), here's a class which enumerates for a random amount of time - basically it keeps going until the next random number from 0-19 is 0:

public class RandomEnumerable : IEnumerable<int>

{

   public IEnumerator<int> GetEnumerator()

   {

       Random rng = new Random();

       int next;

       while ((next=rng.Next(20)) != 0)

       {

           yield return next;

       }

   }

   IEnumerator IEnumerable.GetEnumerator()

   {

       return GetEnumerator();

   }

}

This enumerable *can* be used with SmartEnumerable with no problems - the output is effectively just delayed a little bit.

Jon

Wednesday, August 01, 2007 3:03 PM by skeet

# re: Smart enumerations

Why did you make Entry a class? Given its nature I would have thought a struct would be better from a performance standpoint.

Friday, December 14, 2007 11:50 PM by James Newton-King

# re: Smart enumerations

Yes, in retrospect it probably should be a struct.

Saturday, December 15, 2007 2:30 AM by skeet