Concern

I’ve been enjoying using C# recently, for the most part. The Mono community in particular is great; the caliber of people is almost as good as I saw around GNU Classpath, and they do seem to have a more cohesive direction than the free Java community did, before OpenJDK (and maybe still…). I have some worries that they’re pushing things out the door half-baked, and aren’t telling you about it — the fact that they ship stubs grates against me, and the MonoTODO property is not a practical answer.

The best part of C# is that in places Microsoft did see fit to just say, fuck it, and add in things that aren’t the clean paradigm that Java wanted to keep, because it’s so practical to have lower-level and unsafe access available. I also like the low barrier-to-entry for remoting, and the ability to pass around function pointers and use lambda expressions.

Which got me thinking, what could we do to enable remote lambda expressions? That is, can we do something like this:

Widget w = (Widget) Activator.GetObject(...);
int[] x = new int[] { ... };
int[] y = w.Map (x, i => i * i);

working, where the Widget we’re using is a remote object? It turns out that this is possible, though maybe not easy, with C# as it is [1]. You can’t serialize a lambda expression (probably, it’s a System.Func, but I’m not clear on the internals of this), though you could reference the generated function, which is how functions get passed normally, but that requires that the function be “well known” on both sides of the connection, so you’d have to know about the same lambda expression everywhere,

That’s not what we’re after. We want instant grid computing — just write code, send it over the wire, have it run. You don’t need to update the software on the server.

You can get the System.Reflection.MethodInfo for the expression from the System.Delegate.Method property, then get the System.Reflection.MethodBody for this method. This has the method GetILAsByteArray, which gives you the actual Intermediate Language for that method, and properties for getting the exception and local variable tables. You also have System.Reflection.Emit.DynamicMethod and DynamicILInfo, by which you can generate a method on the fly from the raw IL code and associated metadata. So, you can yank out the relevant method info out of the method, serialize it (it’s now just a byte array and some simple structures; even if the objects themselves aren’t serializable, it’s easy enough to distill it down to something you can serialize — you’d need a custom formatter for that, but hey, have one) send it over the wire, deserialize it, rebuild the method, then run it.

This is surprisingly easy with lambda expressions that don’t need closures, and are dealing with simple data types. I’ve been able to get my dumb little square example to run, using my s-expression stuff, on Mono SVN.

If you are going to want to use closures — which means another remote object (”this”) but going in the other direction! — or call any methods or use strings, things get more complicated. The latter uses tokens, not portable identifiers, to reference things, and these tokens might not be portable between virtual machines (one might not even exist on the other side). One could manage all this state, but it’s a non-trivial thing to do.

Something like this sounds really awesome, to me. You get the flexibility of an embedded scripting language, but your dynamic code is completely embedded into your main code (not a separate script file or — ick — a string) and you get full, optimizing JIT compilation.

1. Note that Mono doesn’t have support for this yet, but I’ve been hacking on this, and might get around to implementing it fully.