Making reflection fly and exploring delegates

re: Making reflection fly and exploring delegates

Kenneth Xu — Tue, 19 May 2009 03:57:26 GMT

Great post! found this when I was surprised by my own test result. You article confirm this for me.

In terms of the generic part, there are two different workarounds that you get return the strong typed delegate instead of general Delegate type. For detail see kennethxu.blogspot.com/.../strong-typed-high-performance.html

I have created a library that provides extension methods so you can simply do this:

var indexOfFunc = typeof(string).GetInstanceInvoker<Func<string, char, int>>("IndexOf");

int index = indexOfFunc("Hello", 'e');

Cheers!

re: Making reflection fly and exploring delegates

Ajai Shankar — Mon, 23 Feb 2009 14:51:34 GMT

Jon

I recently came across this post while exploring a similar implementation.

Have released a multimethod dispatcher that utilizes this technique for invoking reflected methods.

www.codeplex.com/multimethods

Cheers

Ajai

Functions as return values | keyongtech

Functions as return values | keyongtech — Sun, 18 Jan 2009 16:44:16 GMT

Pingback from Functions as return values | keyongtech

re: Making reflection fly and exploring delegates

Mikhail Ryagin — Thu, 20 Nov 2008 18:02:14 GMT

Even if you don't care about performance, consider using delegate over MethodInfo.Invoke() because delegate does not wrap exceptions in

TargetInvocationException. Unwrapping exceptions from TargetInvocationException can be a real pain especially when you don't want to loose the stacktrace (see stackoverflow.com/.../in-c-how-can-i-rethrow-innerexception-without-losing-stack-trace).

re: Making reflection fly and exploring delegates

Kelvin Li — Thu, 04 Sep 2008 19:14:21 GMT

Jon, sorry it's a bug on my side. I was testing a couple of reflection methods and passed (new object[]{new char[]{'a'}}) to the delegate instead of (new char[]{'a'}).

re: Making reflection fly and exploring delegates

skeet — Thu, 04 Sep 2008 18:38:31 GMT

@Kelvin: Could you say exactly what's failing for you? It's not clear what you're trying to do, exactly.

Here's the equivalent Main method calling MagicHelper (unchanged) for TrimEnd:

static void Main()

{

MethodInfo trimEnd = typeof(string).GetMethod("TrimEnd", new Type[]{typeof(char[])});

Func<string, object, object> trimEndFunc = MagicMethod<string>(trimEnd);

Console.WriteLine(trimEndFunc("teststring", new char[]{'a'}));

Console.WriteLine(trimEndFunc("teststring", new char[]{'n', 'g'}));

}

If you want to mail me (skeet@pobox.com) any failing code, that might work better than doing it here.

Jon

re: Making reflection fly and exploring delegates

Kelvin Li — Thu, 04 Sep 2008 18:12:42 GMT

well, this works as long as the parameters are not array. For example, using this method to call "teststring".TrimEnd(new char[] {'a'}) with give an error, saying cannot convert 'System.Object[]' to 'System.Char[]'. Any cure for this?

re: Making reflection fly and exploring delegates

Jorge Varas — Wed, 20 Aug 2008 17:34:31 GMT

GREAT!!!

Now it works. Here is the full example, including the Correction to the lamda and the declaration of Func:

using System;

using System.Reflection;

using System.Text;

public class Test {

public delegate W Func<T, V, W>( T arg, V val );

static void Main( ) {

MethodInfo indexOf = typeof( string ).GetMethod( "IndexOf", new Type[ ] { typeof( char ) } );

MethodInfo getByteCount = typeof( Encoding ).GetMethod( "GetByteCount", new Type[ ] { typeof( string ) } );

Func<string, object, object> indexOfFunc = MagicMethod<string>( indexOf );

Func<Encoding, object, object> getByteCountFunc = MagicMethod<Encoding>( getByteCount );

Console.WriteLine( indexOfFunc( "Hello", 'e' ) );

Console.WriteLine( getByteCountFunc( Encoding.UTF8, "Euro sign: \u20ac" ) );

Console.ReadKey( );

}

static Func<T, object, object> MagicMethod<T>( MethodInfo method ) where T: class {

// First fetch the generic form

MethodInfo genericHelper = typeof( Test ).GetMethod( "MagicMethodHelper",

BindingFlags.Static | BindingFlags.NonPublic );

// Now supply the type arguments

MethodInfo constructedHelper = genericHelper.MakeGenericMethod

( typeof( T ), method.GetParameters( )[ 0 ].ParameterType, method.ReturnType );

// Now call it. The null argument is because it's a static method.

object ret = constructedHelper.Invoke( null, new object[ ] { method } );

// Cast the result to the right kind of delegate and return it

return ( Func<T, object, object> )ret;

}

static Func<TTarget, object, object> MagicMethodHelper<TTarget, TParam, TReturn>( MethodInfo method )

where TTarget: class {

// Convert the slow MethodInfo into a fast, strongly typed, open delegate

Func<TTarget, TParam, TReturn> func = ( Func<TTarget, TParam, TReturn> )Delegate.CreateDelegate

( typeof( Func<TTarget, TParam, TReturn> ), method );

// Now create a more weakly typed delegate which will call the strongly typed one

Func<TTarget, object, object> ret = delegate( TTarget target, object param ) { return func( target, ( TParam )param ); };

return ret;

}

re: Making reflection fly and exploring delegates

skeet — Wed, 20 Aug 2008 16:34:45 GMT

@Varas: You just need to change the lambda expressions into anonymous methods. For instance change the lambda expression in the example into:

Func<TTarget, object, object> ret = delegate(TTarget target, object param)

{ return func(target, (TParam) param); };

Hope this helps,

Jon

re: Making reflection fly and exploring delegates

Jorge Varas — Wed, 20 Aug 2008 16:26:59 GMT

Hi Jon,

Great article... but I have a problem.

I ned to use this idea in .Net 2.0... as in I don't have access to lambda expressions. (I was surpriced at your comment to Vladimir).

How would this be resolved without using lambda expressions?

re: Making reflection fly and exploring delegates

Keith J. Farmer — Fri, 15 Aug 2008 08:00:50 GMT

I'm got some code lying around where I created some strong-typed wrappers around the reflection types, to denote such things as declaring class, data type of property, etc.

It was an interesting exercise.

re: Making reflection fly and exploring delegates

Vlad — Thu, 14 Aug 2008 21:48:36 GMT

Nice writeup. An old MSDN article by Joel Pobar (CLR PM) found here: msdn.microsoft.com/.../cc163759.aspx talks about similar techniques, and also includes information about dynamically generating code through DynamicMethod to improve performance.

re: Making reflection fly and exploring delegates

skeet — Mon, 11 Aug 2008 08:54:24 GMT

Vladimir: Yes, expression trees can make this a lot simpler. Unfortunately I'm targeting .NET 2.0...

Jon

re: Making reflection fly and exploring delegates

Vladimir Matveev — Mon, 11 Aug 2008 08:32:25 GMT

Hello Jon.

I think, that usage of Expressions can make solution move obvious and "expressive" :)

public static void Main(string[] args)

{

var indexOf = typeof(string).GetMethod("IndexOf", new[] { typeof(char) });

var getByteCount = typeof(Encoding).GetMethod("GetByteCount", new[] { typeof(string) });

var indexOfFunc = MagicMethod<string>(indexOf);

var getByteCountFunc = MagicMethod<Encoding>(getByteCount);

Console.WriteLine(indexOfFunc("Hello", 'e'));

Console.WriteLine(getByteCountFunc(Encoding.UTF8, "Euro sign: \u20ac"));

}

static Func<T, object, object> MagicMethod<T>(MethodInfo method)

{

var parameter = method.GetParameters().Single();

var instance = Expression.Parameter(typeof (T), "instance");

var argument = Expression.Parameter(typeof (object), "argument");

var methodCall = Expression.Call(

instance,

method,

Expression.Convert(argument, parameter.ParameterType)

);

return Expression.Lambda<Func<T, object, object>>(

Expression.Convert(methodCall, typeof (object)),

instance, argument

).Compile();

}

re: Making reflection fly and exploring delegates

skeet — Sat, 09 Aug 2008 20:42:25 GMT

Anders: I haven't actually gone down the manual IL emission route yet, but I've certainly considered it. However, I wonder what the memory implications might be. In particular, part of the point of using "optimise for size" is to avoid creating too much code for the JITter etc. I have to admit that I don't know what the memory implications of using even the technique I've got at the moment might be.

To be honest I would really suggest that for almost *all* situations "optimise for speed" is far more appropriate for Protocol Buffers. I don't know why it's not the default :(

At the moment the bottleneck isn't in the reflection itself, so I probably won't try IL emission in this particular case... but sooner or later I'm bound to want it :)

Jon

Dynamic creation of delegates using IL

Anders Borum — Sat, 09 Aug 2008 20:20:41 GMT

Hi Jon,

A great article on optimization - it's been a while since you've been writing on this topic.

While I agree that the complexity of this implementation is quite high, it's important to realize that this pattern (and related implementations) is becoming more common these days - meaning that more developers would be able to step up to the task of maintaining and evolving the code.

The number of applications that can benefit from this is increasing rapidly; especially with APIs in e.g. content management applications that supports untyped lists / tables (think SharePoint and so on) where dynamic creation of fields (that are not know at compile time) requires a really fast implementation.

The performance of the untyped lists (and thus their overall benefit and succes) directly relates to the performance of the underlying API.

I'm not familiar with the implementation of the Protocol Buffers projects, but if there's a requirement for calling a known method on a dynamic type, you could benefit from dynamic IL emission and cache the concrete delegate.

In the CMS API I'm working on, one of the requirements is to do just that; call a known method (in this case a constructor) on a dynamic type (a lot of examples like this is starting to show up these days - it's almost like a trend). There's a lot of performance in code like this, but you have to stay sharp and document every step thoroughly.

Here's the part of the framework I was referring to above, namely the dynamic creation of CmsField instances (that goes into the x/y coordinate of a row/column) by means of creating a concrete delegate for each dynamic type encountered during runtime.

Because of optimizations like this, all types (not just internals) become first class citizens in terms of performance (100.000s per sec.) - I really like that way of thinking.

private static Dictionary<string, FieldDelegate> _cache = new Dictionary<string, FieldDelegate>();

private delegate CmsField FieldDelegate(CmsContext cms);

public static CmsField GetFieldDelegate(CmsContext cms, string typeName)

{

if (typeName == null) { throw new ArgumentNullException("typeName"); }

FieldDelegate result = null;

if (_cache.TryGetValue(typeName, out result)) { return result(cms); }

lock (_cache)

{

if (_cache.TryGetValue(typeName, out result)) { return result(cms); }

Type type = Type.GetType(typeName);

if (type == null) { throw new InvalidOperationException(string.Format("Unable to find the type '{0}'.", typeName)); }

ConstructorInfo info = type.GetConstructor(new Type[] { typeof(CmsContext) });

if (info == null) { throw new InvalidOperationException(string.Format("Unable to find a public constructor with the signature '{0}(CmsContext)' for type {1}", type.Name, typeName)); }

var method = new DynamicMethod("", typeof(CmsField), new Type[] { typeof(CmsContext) }, type);

ILGenerator il = method.GetILGenerator();

il.Emit(OpCodes.Ldarg_0);

il.Emit(OpCodes.Newobj, info);

il.Emit(OpCodes.Ret);

_cache.Add(typeName, result = (FieldDelegate) method.CreateDelegate(typeof(FieldDelegate)));

}

return result(cms);

}

I'm sure you've ventured down this path before, but it would be interesting to see your take on thoughts such as this (strictly related to performance).

re: Making reflection fly and exploring delegates

Marc Gravell — Sat, 09 Aug 2008 20:05:11 GMT

Good write up; I use the same trick (especially the open delegate to call an instance method) quite frequently, but with an additional micro-optimisation; by making the *caller* generic, you can avoid the cast and (potentially) box/unbox. But this is nowhere as signifcant as the difference simply from using a typed delegate (rather than just Delegate or MethodInfo).

Unfortunately, having a generic caller adds another dimension and tend to involve yet more MakeGeneric[Method|Type]... but "every little helps" ;-p