Closures - Revision history

JaroslavTulach at 10:18, 5 March 2014

JaroslavTulach — Wed, 05 Mar 2014 10:18:42 GMT

←Older revision		Revision as of 10:18, 5 March 2014
Line 1:		Line 1:
-	[[wikipedia::Closure_(computer_science)\|Closures]] are classical [[OOP]] approach to represent a block of code that can be invoked passing in few parameters. [[Closures]] are typical building block of [[Smalltalk]] systems. Origin of closures (called lamdas back then) can however be traced into pre-historic age of computer science. The [[λ-calculus]] in fact knows nothing else than invocable blocks of code.	+	[[wikipedia::Closure_(computer_science)\|Closures]] (nicknamed ''lambdas'' in [[JDK]]8) are classical [[OOP]] approach to represent a block of code that can be invoked passing in few parameters. [[Closures]] are typical building block of [[Smalltalk]] systems. Origin of closures (called lamdas back then) can however be traced into pre-historic age of computer science. The [[λ-calculus]] in fact knows nothing else than invocable blocks of code.

	Since their invention [[closures]] become almost mandatory syntactic element for any new language. Not having them become a ''faux pas''. As the profound [[Blogs:JaroslavTulach:Theory:History_of_Programming_Languages\|history of programming languages]] puts it: ''[[Java]] makes them popular by not having them''.		Since their invention [[closures]] become almost mandatory syntactic element for any new language. Not having them become a ''faux pas''. As the profound [[Blogs:JaroslavTulach:Theory:History_of_Programming_Languages\|history of programming languages]] puts it: ''[[Java]] makes them popular by not having them''.

JaroslavTulach at 09:39, 1 April 2012

JaroslavTulach — Sun, 01 Apr 2012 09:39:34 GMT

←Older revision		Revision as of 09:39, 1 April 2012
Line 3:		Line 3:
	Since their invention [[closures]] become almost mandatory syntactic element for any new language. Not having them become a ''faux pas''. As the profound [[Blogs:JaroslavTulach:Theory:History_of_Programming_Languages\|history of programming languages]] puts it: ''[[Java]] makes them popular by not having them''.		Since their invention [[closures]] become almost mandatory syntactic element for any new language. Not having them become a ''faux pas''. As the profound [[Blogs:JaroslavTulach:Theory:History_of_Programming_Languages\|history of programming languages]] puts it: ''[[Java]] makes them popular by not having them''.

-	Looks like things are about to change. [[Sun]] recently announced its will to implement [[closures]] for [[JDK]]7. To join the overwhelming ecstasy in the [[Java]] community I decided to write this page and provide few insights from less conventional angles.	+	Looks like things are about to change. [[Sun]] recently announced its will to implement [[closures]] for [[JDK]]7 (now corrected by [[Oracle]] to be [[JDK]]8). To join the overwhelming ecstasy in the [[Java]] community I decided to write this page and provide few insights from less conventional angles.

	== The Da Vinci Closures ==		== The Da Vinci Closures ==

JaroslavTulach at 09:55, 1 December 2009

JaroslavTulach — Tue, 01 Dec 2009 09:55:20 GMT

←Older revision		Revision as of 09:55, 1 December 2009
Line 15:		Line 15:
	Now when [[Sun]] adopted the idea of [[closures]] for [[JDK]]7, everything is clear. There was a hidden agenda behind the publicly stated goal. Let me take you through the plot plan!		Now when [[Sun]] adopted the idea of [[closures]] for [[JDK]]7, everything is clear. There was a hidden agenda behind the publicly stated goal. Let me take you through the plot plan!

-	~~=== [[InvokeDynamic\|Method Handle]] ===~~	+	{{:ClosuresAndMethodHandles}}
-		+
-	{{:~~InvokeDynamic~~}}	+
-		+
-	~~=== [[Closures]] as innerclasses ===~~	+
-		+
-	The typical expectation for implementing [[closures]] (for example the [http://javac.info/closures-v06a.html 0.6a version]) seems to envision a closure as an innerclass (if it says at all, how [[closures]] shall be implemented), with simplified syntax. This is indeed possible, yet ineffective. Overhead of defining new (inner) class in [[Java]] is high. Each class occupies a single .class file and these files are selfcontained. They contain not only their code, but also all their static linking information (e.g. the constant pool). This information gets copied with each inner class. Splitting one class into three does not keep the final size proportional to the original one. Imagine you want to rewrite following code:	+
-		+
-	~~<source lang="java">~~	+
-	~~class SayHello {~~	+
-	~~public void sayHello(String to) {~~	+
-	~~String hello = "hello";~~	+
-		+
-	~~System.out.println(hello);~~	+
-	~~System.out.println(to);~~	+
-	}	+
-	}	+
-	~~</source>~~	+
-		+
-	~~so that each of the printlns runs under some lock (let's expect there there is some static method withLock(Runnable) and that we can use some form of [[closures]]):~~	+
-		+
-	~~<source lang="java">~~	+
-	~~class SayHelloSafely {~~	+
-	~~public void sayHello(String to) {~~	+
-	~~String hello = "hello";~~	+
-		+
-	~~withLock({ System.out.println(hello); });~~	+
-	~~withLock({ System.out.println(to); });~~	+
-	}	+
-	}	+
-	~~</source>~~	+
-		+
-	Due to power of [[closures]] this code is as simple as the original one (just the call to ''withLock'' is added, but that was intended change to satisfy our goal). However if we stick with the originally planned implementation of [[closures]] as inner classes, then the above code in fact means:	+
-		+
-	~~<source lang="java">~~	+
-	~~class SayHelloSafely {~~	+
-	~~public void sayHello(final String to) {~~	+
-	~~final String hello = "hello";~~	+
-	~~withLock(new Runnable() {~~	+
-	~~public void run() {~~	+
-	~~System.out.println(hello);~~	+
-	}	+
-	~~});~~	+
-	~~withLock(new Runnable() {~~	+
-	~~public void run() {~~	+
-	~~System.out.println(to);~~	+
-	}	+
-	~~});~~	+
-	}	+
-	}	+
-	~~</source>~~	+
-		+
-	Even this simple example shows how ineffective trivial implementation of [[closures]] can be. Instead of one class, we have three. Each of them having significant overlaps in their constant pools. Given the expected proliferation of closure based [[API]]s (as they are easy to use, much easier than innerclasses), this can lead to enormous and unnecessary waste of memory. As one who watches over [[performance]] of [[NetBeans]] I cannot silently let this happen.	+
-		+
-	~~=== The Mapping ===~~	+
-		+
-	Thankfully there is a cure. It is possible to write well performing implementation of [[closures]] using [[invokeDynamic]] and its method handles. Imagine that the above code is rewritten to use method handles (and that the withLock method now takes MethodHandle):	+
-		+
-	~~<source lang="java">~~	+
-	~~class SayHelloEffectively {~~	+
-	~~private static MethodHandle first;~~	+
-	~~private static MethodHandle second;~~	+
-		+
-	~~public void sayHello(final String to) {~~	+
-	~~MethodHandle addThis = MethodHandles.insertArgument(first, 0, this);~~	+
-	~~withLock(first);~~	+
-	~~MethodHandle applyToAndThis =~~	+
-	~~MethodHandle.insertArgument(MethodHandles.insertArgument(second, 0, to), 0, this);~~	+
-	~~withLock(second);~~	+
-	}	+
-		+
-	~~private void firstRunnable() {~~	+
-	~~final String hello = "hello";~~	+
-	~~System.out.println(hello);~~	+
-	}	+
-	~~private void secondRunnable(String to) {~~	+
-	~~System.out.println(to);~~	+
-	}	+
-		+
-	~~static {~~	+
-	~~first = MethodHandles.lookup().findSpecial(~~	+
-	~~SayHelloEffectively.class, "firstRunnable",~~	+
-	~~MethodHandles.methodType(void.class)~~	+
-	);	+
-	~~second = MethodHandles.lookup().findSpecial(~~	+
-	~~SayHelloEffectively.class, "secondRunnable",~~	+
-	~~MethodHandles.methodType(void.class, String.class)~~	+
-	);	+
-	}	+
-	}	+
-	~~</source>~~	+
-		+
-	Please accept my appology for the above use of method handling [[API]]. It is just a sketch of the implementation. I have not found the javadoc to verify or even compile my code against it. Anyway it is clear that this conversion of [[closures]] is very constant pool friendly. Regardless of the amount of [[closures]] in a class, just one, shared constant pool is used. This avoids useless duplication of its entries in the inner classes.	+
-		+
-	The method handle solution shall also be well performant. Method handle combinators are supposed to be effective. The only slow operation is the reflective binding, but that happens just once, when the class is loaded (or the method is first used).	+
-		+
-	Also notice that this approach really supports [[closures]]. If a piece of code references some variable from outer block, it is easy (because of method handle combinators) to pass such variable into the closure method via an argument. Sort of like partially applied functions in high level languages.	+
-		+
-	Also, in all the [[closures]] for [[Java]] specifications, the '''this''' is treated differently than in inner classes. It is supposed to mean the outer class, which would require certain compiler transformations if [[closures]] were implemented as hidden inner classes. If [[closures]] are implemented as method handles, the meaning of '''this''' naturally stays the same, as the methods are really methods of the proper class.	+

	== Declination ==		== Declination ==

JaroslavTulach: /* Declination */

JaroslavTulach — Sun, 29 Nov 2009 08:26:28 GMT

Declination

←Older revision		Revision as of 08:26, 29 November 2009
Line 117:		Line 117:
	== Declination ==		== Declination ==

-	At the beginning I accused [[Sun]] for having a hidden agenda. I do not mean it. Given all the problems the [[Sun]]'s [[JDK]] team had to keep even to simple plans after open sourcing [[JDK]], I cannot imagine how it could execute something as complex. Pretending that [[invokeDynamic]] is for other languages on top of [[HotSpot]] virtual machine and doing all of this just because of [[closures]] is too complex to be real. Also, when I explained the usefulness of [[invokeDynamic]] for [[closures]] to Peter von Ahe few years ago (when he was working for [[Sun]] and responsible for the [[JavaC]] compiler), he was pleasantly surprised and slightly interested in the topic. So I do not think the compiler team knew about the emerging effects of these two parallel efforts. Anyway this is not that interesting.	+	At the beginning I accused [[Sun]] for having a hidden agenda. I do not mean it. Given all the problems the [[Sun]]'s [[JDK]] team had to keep even to simple plans after open sourcing [[JDK]], I cannot imagine how it could execute something as complex. Pretending that [[invokeDynamic]] is for other languages on top of [[HotSpot]] virtual machine and doing all of this just because of [[closures]] is too complex to be real. Also, when I explained the usefulness of [[invokeDynamic]] for [[closures]] to Peter von Ahe few years ago (when he was working for [[Sun]] and responsible for the [[JavaC]] compiler), he was pleasantly surprised and slightly interested in the topic. I do not think the compiler team knew about the emerging effects of these two parallel efforts. Anyway this is not that interesting.

	What is important is that there is an effective way to implement [[closures]] for [[Java]]. It is also good that most of the work (on runtime performance) has already been done by the ''Da Vinci Machine'' project. And last, but not least - it is very good that the same infrastructure used by non-[[Java]] languages on top of [[JVM]] will now be shared by core [[Java]]. This will make it a primary focus for wider groups of engineers making the [[invokeDynamic]] and method handle combinators more and more effective in the future.		What is important is that there is an effective way to implement [[closures]] for [[Java]]. It is also good that most of the work (on runtime performance) has already been done by the ''Da Vinci Machine'' project. And last, but not least - it is very good that the same infrastructure used by non-[[Java]] languages on top of [[JVM]] will now be shared by core [[Java]]. This will make it a primary focus for wider groups of engineers making the [[invokeDynamic]] and method handle combinators more and more effective in the future.

	<comments/>		<comments/>

JaroslavTulach: /* Declination */

JaroslavTulach — Sun, 29 Nov 2009 08:25:47 GMT

Declination

←Older revision		Revision as of 08:25, 29 November 2009
Line 117:		Line 117:
	== Declination ==		== Declination ==

-	At the beginning I accused [[Sun]] for having a hidden agenda. I do not mean it. Given all the problems the [[Sun]]'s [[JDK]] team had to keep even to simple plans after open sourcing [[JDK]], I cannot imagine how ~~could~~ it execute something as complex. Pretending that [[invokeDynamic]] is for other languages on top of [[HotSpot]] virtual machine and doing all of this just because of [[closures]] is too complex to be real. Also, when I explained the usefulness of [[invokeDynamic]] for [[closures]] to Peter von Ahe few years ago (when he was working for [[Sun]] and responsible for the [[JavaC]] compiler), he was pleasantly surprised and slightly interested in the topic. So I do not think the compiler team knew about the emerging effects of these two parallel efforts. Anyway this is not that interesting.	+	At the beginning I accused [[Sun]] for having a hidden agenda. I do not mean it. Given all the problems the [[Sun]]'s [[JDK]] team had to keep even to simple plans after open sourcing [[JDK]], I cannot imagine how it could execute something as complex. Pretending that [[invokeDynamic]] is for other languages on top of [[HotSpot]] virtual machine and doing all of this just because of [[closures]] is too complex to be real. Also, when I explained the usefulness of [[invokeDynamic]] for [[closures]] to Peter von Ahe few years ago (when he was working for [[Sun]] and responsible for the [[JavaC]] compiler), he was pleasantly surprised and slightly interested in the topic. So I do not think the compiler team knew about the emerging effects of these two parallel efforts. Anyway this is not that interesting.

	What is important is that there is an effective way to implement [[closures]] for [[Java]]. It is also good that most of the work (on runtime performance) has already been done by the ''Da Vinci Machine'' project. And last, but not least - it is very good that the same infrastructure used by non-[[Java]] languages on top of [[JVM]] will now be shared by core [[Java]]. This will make it a primary focus for wider groups of engineers making the [[invokeDynamic]] and method handle combinators more and more effective in the future.		What is important is that there is an effective way to implement [[closures]] for [[Java]]. It is also good that most of the work (on runtime performance) has already been done by the ''Da Vinci Machine'' project. And last, but not least - it is very good that the same infrastructure used by non-[[Java]] languages on top of [[JVM]] will now be shared by core [[Java]]. This will make it a primary focus for wider groups of engineers making the [[invokeDynamic]] and method handle combinators more and more effective in the future.

	<comments/>		<comments/>

JaroslavTulach: /* The Da Vinci Closures */

JaroslavTulach — Sun, 29 Nov 2009 08:20:12 GMT

The Da Vinci Closures

←Older revision		Revision as of 08:20, 29 November 2009
Line 13:		Line 13:
	That is why I was quite surprised that the attempt to extend [[Java]] [[bytecode]] with [[invokeDynamic]] has been named [http://openjdk.java.net/projects/mlvm/ Da Vinci Machine] project. What does that mean? Is the proclaimed goal to support multiple languages just a layered public statement and the real goal is somewhere else? What [[Sun]] wants public to think? Why?		That is why I was quite surprised that the attempt to extend [[Java]] [[bytecode]] with [[invokeDynamic]] has been named [http://openjdk.java.net/projects/mlvm/ Da Vinci Machine] project. What does that mean? Is the proclaimed goal to support multiple languages just a layered public statement and the real goal is somewhere else? What [[Sun]] wants public to think? Why?

-	Now when [[Sun]] adopted the idea of [[closures]] for [[JDK]]7, everything is clear. There was a hidden agenda behind the publicly stated goal. Let me ~~show~~ you the ~~whole~~ plot ~~now.~~	+	Now when [[Sun]] adopted the idea of [[closures]] for [[JDK]]7, everything is clear. There was a hidden agenda behind the publicly stated goal. Let me take you through the plot plan!

	=== [[InvokeDynamic\|Method Handle]] ===		=== [[InvokeDynamic\|Method Handle]] ===

JaroslavTulach: /* The Mapping */

JaroslavTulach — Sat, 28 Nov 2009 21:02:50 GMT

The Mapping

←Older revision		Revision as of 21:02, 28 November 2009
Line 95:		Line 95:

	static {		static {
-	first = MethodHandles.~~forMethod~~(SayHelloEffectively.class, "firstRunnable");	+	first = MethodHandles.lookup().findSpecial(
-	second = MethodHandles.~~forMethod~~(SayHelloEffectively.class, "secondRunnable");	+	SayHelloEffectively.class, "firstRunnable",
		+	MethodHandles.methodType(void.class)
		+	);
		+	second = MethodHandles.lookup().findSpecial(
		+	SayHelloEffectively.class, "secondRunnable",
		+	MethodHandles.methodType(void.class, String.class)
		+	);
	}		}
	}		}

JaroslavTulach: /* Declination */

JaroslavTulach — Sat, 28 Nov 2009 20:22:46 GMT

Declination

←Older revision		Revision as of 20:22, 28 November 2009
Line 111:		Line 111:
	== Declination ==		== Declination ==

-	At the beginning I accused [[Sun]] for having a hidden agenda. I do not mean it. Given all the problems the [[Sun]]'s [[JDK]] team had to keep even to simple plans after open sourcing [[JDK]], I cannot imagine how could it execute something as complex ~~as this~~. Pretending that [[invokeDynamic]] is for other languages on top of [[HotSpot]] virtual machine and doing all of this just because of [[closures]] is too complex to be real. Also, when I explained the usefulness of [[invokeDynamic]] for [[closures]] to Peter von Ahe few years ago (when he was working for [[Sun]] and responsible for the [[JavaC]] compiler), he was pleasantly surprised and slightly interested in the topic. So I do not think the compiler team knew about the emerging effects of these two parallel efforts.	+	At the beginning I accused [[Sun]] for having a hidden agenda. I do not mean it. Given all the problems the [[Sun]]'s [[JDK]] team had to keep even to simple plans after open sourcing [[JDK]], I cannot imagine how could it execute something as complex. Pretending that [[invokeDynamic]] is for other languages on top of [[HotSpot]] virtual machine and doing all of this just because of [[closures]] is too complex to be real. Also, when I explained the usefulness of [[invokeDynamic]] for [[closures]] to Peter von Ahe few years ago (when he was working for [[Sun]] and responsible for the [[JavaC]] compiler), he was pleasantly surprised and slightly interested in the topic. So I do not think the compiler team knew about the emerging effects of these two parallel efforts. Anyway this is not that interesting.

-	~~Anyway this is not that interesting.~~ What is important is that there is a effective way to implement [[closures]] for [[Java]]. It is also good that most of the work (on runtime performance) has already been done by the ''Da Vinci Machine'' project. And last, but not least - it is very good that the same infrastructure used by non-[[Java]] languages on top of [[JVM]] will now be shared by core [[Java]]. This will make it a primary focus for wider groups of engineers ~~make~~ the [[invokeDynamic]] and method handle combinators more and more effective in the future.	+	What is important is that there is an effective way to implement [[closures]] for [[Java]]. It is also good that most of the work (on runtime performance) has already been done by the ''Da Vinci Machine'' project. And last, but not least - it is very good that the same infrastructure used by non-[[Java]] languages on top of [[JVM]] will now be shared by core [[Java]]. This will make it a primary focus for wider groups of engineers making the [[invokeDynamic]] and method handle combinators more and more effective in the future.

	<comments/>		<comments/>

JaroslavTulach: /* Closures as innerclasses */

JaroslavTulach — Sat, 28 Nov 2009 20:17:45 GMT

Closures as innerclasses

←Older revision		Revision as of 20:17, 28 November 2009
Line 21:		Line 21:
	=== [[Closures]] as innerclasses ===		=== [[Closures]] as innerclasses ===

-	The typical expectation for implementing closures (for example the [http://javac.info/closures-v06a.html 0.6a version]) seems to envision a closure as an innerclass, with simplified syntax. This is indeed possible, yet ineffective. Overhead of defining new (inner) class in [[Java]] is high. Each class occupies a single .class file and these files are selfcontained. They contain not only their code, but also all their static linking information (e.g. the constant pool). This information gets copied with each inner class. Splitting one class into three does not keep the final size proportional to the original one. Imagine you want to rewrite following code:	+	The typical expectation for implementing [[closures]] (for example the [http://javac.info/closures-v06a.html 0.6a version]) seems to envision a closure as an innerclass (if it says at all, how [[closures]] shall be implemented), with simplified syntax. This is indeed possible, yet ineffective. Overhead of defining new (inner) class in [[Java]] is high. Each class occupies a single .class file and these files are selfcontained. They contain not only their code, but also all their static linking information (e.g. the constant pool). This information gets copied with each inner class. Splitting one class into three does not keep the final size proportional to the original one. Imagine you want to rewrite following code:

	<source lang="java">		<source lang="java">

JaroslavTulach: /* The Da Vinci Closures */

JaroslavTulach — Sat, 28 Nov 2009 20:12:03 GMT

The Da Vinci Closures

←Older revision		Revision as of 20:12, 28 November 2009
Line 13:		Line 13:
	That is why I was quite surprised that the attempt to extend [[Java]] [[bytecode]] with [[invokeDynamic]] has been named [http://openjdk.java.net/projects/mlvm/ Da Vinci Machine] project. What does that mean? Is the proclaimed goal to support multiple languages just a layered public statement and the real goal is somewhere else? What [[Sun]] wants public to think? Why?		That is why I was quite surprised that the attempt to extend [[Java]] [[bytecode]] with [[invokeDynamic]] has been named [http://openjdk.java.net/projects/mlvm/ Da Vinci Machine] project. What does that mean? Is the proclaimed goal to support multiple languages just a layered public statement and the real goal is somewhere else? What [[Sun]] wants public to think? Why?

-	Now when [[Sun]] adopted the idea of [[closures]] for [[JDK]]7, everything is clear. There was a hidden agenda behind the publicly stated goal.	+	Now when [[Sun]] adopted the idea of [[closures]] for [[JDK]]7, everything is clear. There was a hidden agenda behind the publicly stated goal. Let me show you the whole plot now.

	=== [[InvokeDynamic\|Method Handle]] ===		=== [[InvokeDynamic\|Method Handle]] ===