JaroslavTulach: /* Evolution (Versioning, Deprecation) */ - 2011-05-17 10:27:22

Evolution (Versioning, Deprecation)

←Older revision Revision as of 10:27, 17 May 2011
Line 161: Line 161:
DSL versioning is a very powerful concept. Unlike in a Java API, we can completely remove concepts from the latest version of the language instead of deprecating them, keeping the semantics of the language clear and simple. We do this because the old parser logic is still there, and old code will always use the old logic.
DSL versioning is a very powerful concept. Unlike in a Java API, we can completely remove concepts from the latest version of the language instead of deprecating them, keeping the semantics of the language clear and simple. We do this because the old parser logic is still there, and old code will always use the old logic.
 +
 +
''Jarda'': Additional note here is that this requires existence of a shared abstract binary interface behind the language. All the versions of the language then need to compile to the same ''ABI''. If so, then each unit of compilation can be written in different flavor of the language. However the shared ''ABI'' is crucial - I remember example of a language designed without ''ABI'' and problems with its [[evolution]] (e.g. the complete incompatibility of various versions of Java FX Script).

JaroslavTulach: /* DSL examples */ - 2011-05-17 10:22:46

DSL examples

←Older revision Revision as of 10:22, 17 May 2011
Line 23: Line 23:
:* Or the format is close to [[HTML]], like in case of [[Docbook]].
:* Or the format is close to [[HTML]], like in case of [[Docbook]].
-
Jarda: Does this mean that [[XML]] a [[meta]] [[DSL]] language then? A language to help creation of [[DSL]]s without writing a parser? What other languages like this we can find and where is the boundary? For example sometimes, when using a library written in some highlevel language like [[Scala]], [[Haskell]], [[Clean]], then I feel like using some other, completely different language. To give a [[BNF]] grammar example, compare for example [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.1785&rep=rep1&type=pdf Haskell Parser Combinators]]. Of course you are bound by the underlaying language (but the same applies to [[XML]]), but the program code itself is quite close to [[BNF]] and gets processed automatically without writing anything special. Conclusion? Good language ([[Haskel]]) and good library (the parser generators for example), gives you same comfort more easily.
+
Jarda: Does this mean that [[XML]] a [[meta]] [[DSL]] language then? A language to help creation of [[DSL]]s without writing a parser? What other languages like this we can find and where is the boundary? For example sometimes, when using a library written in some highlevel language like [[Scala]], [[Haskell]], [[Clean]], then I feel like using some other, completely different language. To give a [[BNF]] grammar example, compare for example [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.1785&rep=rep1&type=pdf Haskell Parser Combinators]. Of course you are bound by the underlaying language (but the same applies to [[XML]]), but the program code itself is quite close to [[BNF]] and gets processed automatically without writing anything special. Conclusion? Good language ([[Haskell]]) and good library (the parser generators for example), gives you same comfort more easily.
Rich: Yes, this is essentially correct, though I would draw a distinction between [[XML]] and these other examples. Martin Fowler calls the latter an "[http://martinfowler.com/articles/languageWorkbench.html#InternalDsl internal DSL]". Languages like [[LISP]] are particularly suited to this, though [[Scala]] has been making much hay of it lately. Also, the extreme operator overloading in [[Fortress]] theoretically makes it an excellent foundation for DSLs.
Rich: Yes, this is essentially correct, though I would draw a distinction between [[XML]] and these other examples. Martin Fowler calls the latter an "[http://martinfowler.com/articles/languageWorkbench.html#InternalDsl internal DSL]". Languages like [[LISP]] are particularly suited to this, though [[Scala]] has been making much hay of it lately. Also, the extreme operator overloading in [[Fortress]] theoretically makes it an excellent foundation for DSLs.

JaroslavTulach: /* DSL examples */ - 2011-05-17 10:22:00

DSL examples

←Older revision Revision as of 10:22, 17 May 2011
Line 23: Line 23:
:* Or the format is close to [[HTML]], like in case of [[Docbook]].
:* Or the format is close to [[HTML]], like in case of [[Docbook]].
-
Jarda: Does this mean that [[XML]] a [[meta]] [[DSL]] language then? A language to help creation of [[DSL]]s without writing a parser? What other languages like this we can find and where is the boundary? For example sometimes, when using a library written in some highlevel language like [[Scala]], [[Haskel]], [[Clean]], then I feel like using some other, completely different language. To give a [[BNF]] grammar example, compare for example [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.1785&rep=rep1&type=pdf Haskell Parser Combinators]]. Of course you are bound by the underlaying language (but the same applies to [[XML]]), but the program code itself is quite close to [[BNF]] and gets processed automatically without writing anything special. Conclusion? Good language ([[Haskel]]) and good library (the parser generators for example), gives you same comfort more easily.
+
Jarda: Does this mean that [[XML]] a [[meta]] [[DSL]] language then? A language to help creation of [[DSL]]s without writing a parser? What other languages like this we can find and where is the boundary? For example sometimes, when using a library written in some highlevel language like [[Scala]], [[Haskell]], [[Clean]], then I feel like using some other, completely different language. To give a [[BNF]] grammar example, compare for example [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.1785&rep=rep1&type=pdf Haskell Parser Combinators]]. Of course you are bound by the underlaying language (but the same applies to [[XML]]), but the program code itself is quite close to [[BNF]] and gets processed automatically without writing anything special. Conclusion? Good language ([[Haskel]]) and good library (the parser generators for example), gives you same comfort more easily.
Rich: Yes, this is essentially correct, though I would draw a distinction between [[XML]] and these other examples. Martin Fowler calls the latter an "[http://martinfowler.com/articles/languageWorkbench.html#InternalDsl internal DSL]". Languages like [[LISP]] are particularly suited to this, though [[Scala]] has been making much hay of it lately. Also, the extreme operator overloading in [[Fortress]] theoretically makes it an excellent foundation for DSLs.
Rich: Yes, this is essentially correct, though I would draw a distinction between [[XML]] and these other examples. Martin Fowler calls the latter an "[http://martinfowler.com/articles/languageWorkbench.html#InternalDsl internal DSL]". Languages like [[LISP]] are particularly suited to this, though [[Scala]] has been making much hay of it lately. Also, the extreme operator overloading in [[Fortress]] theoretically makes it an excellent foundation for DSLs.

JaroslavTulach: /* Accessing Database */ - 2011-05-17 10:21:11

Accessing Database

←Older revision Revision as of 10:21, 17 May 2011
Line 100: Line 100:
==== Accessing Database ====
==== Accessing Database ====
-
Jarda: Is it easier in [[DSL]] to access database structure, than in [[Java]]? I would expect so, I would expect that [[DSL]] can make it easy easy as [[Ruby]] on Rails or Grails - e.g. you just access object representing the database and all columns are accessible as fields. As soon as you change the database schema, new field will immediately be visible. This could be hard to simulate when writing [[Java]] libraries, or not?
+
Jarda: Is it easier in [[DSL]] to access database structure, than in [[Java]]? I would expect so, I would expect that [[DSL]] can make it easy easy as [[Ruby]] on Rails or Grails - e.g. you just access object representing the database and all columns are accessible as fields. As soon as you change the database schema, new field will immediately be visible. This could be hard to simulate when writing [[Java]] libraries, or not? Actually not - see [[LiveDB]] demo!
Rich: I would agree with this, but more because of the "Domain" part of "Domain Specific Language" than the "Language" part. The Apex runtime is aware of the database that is available, already has access to a connection pool, and already knows the tables and columns that have been configured by the organization. A library could take advantage of the same information, but it is really runtime information, and so would rely on a custom, domain-specific runtime. Once you have a general purpose language running in a domain specific runtime, you're really 95% of the way to what we did with Apex, which is to say "as long as we've got our own runtime, let's make things a bit easier on our customers by adding some keywords for the really common stuff". That starts you down the road of a full DSL.
Rich: I would agree with this, but more because of the "Domain" part of "Domain Specific Language" than the "Language" part. The Apex runtime is aware of the database that is available, already has access to a connection pool, and already knows the tables and columns that have been configured by the organization. A library could take advantage of the same information, but it is really runtime information, and so would rely on a custom, domain-specific runtime. Once you have a general purpose language running in a domain specific runtime, you're really 95% of the way to what we did with Apex, which is to say "as long as we've got our own runtime, let's make things a bit easier on our customers by adding some keywords for the really common stuff". That starts you down the road of a full DSL.

JaroslavTulach at 09:41, 17 May 2011 - 2011-05-17 09:41:20

←Older revision Revision as of 09:41, 17 May 2011
Line 1: Line 1:
-
[[wikipedia::Domain Specific Language|Domain Specific Language]]s will be topic of [[User:RichUnger]] and [[User:JaroslavTulach]] JavaOne 2010 shootout. A very interesting introductory external resource can be found [http://www.ibm.com/developerworks/java/library/j-eaed13/index.html?ca=drs- here].
+
[[wikipedia::Domain Specific Language|Domain Specific Language]]s will be topic of [[User:RichUnger]] and [[User:JaroslavTulach]] [[JavaOne]] 2010 shootout. A very interesting introductory external resource can be found [http://www.ibm.com/developerworks/java/library/j-eaed13/index.html?ca=drs- here].
= Overview =
= Overview =

JaroslavTulach: /* Overview */ - 2010-10-27 21:23:08

Overview

←Older revision Revision as of 21:23, 27 October 2010
Line 2: Line 2:
= Overview =
= Overview =
 +
 +
We had a presentation about [[DSL]]s during [[JavaOne2010]]. Here are the slides: [[Image:Domain-library-shootout.pdf]].
===What is a [[wikipedia::Domain Specific Language|DSL]]?===
===What is a [[wikipedia::Domain Specific Language|DSL]]?===
Line 73: Line 75:
FORWARD 100
FORWARD 100
LEFT 90
LEFT 90
-
 
= Developer Experience =
= Developer Experience =

JaroslavTulach at 12:42, 15 June 2010 - 2010-06-15 12:42:34

←Older revision Revision as of 12:42, 15 June 2010
Line 1: Line 1:
-
[[wikipedia::Domain Specific Language|Domain Specific Language]]s will be topic of [[User:RichUnger]] and [[User:JaroslavTulach]] JavaOne 2010 shootout.
+
[[wikipedia::Domain Specific Language|Domain Specific Language]]s will be topic of [[User:RichUnger]] and [[User:JaroslavTulach]] JavaOne 2010 shootout. A very interesting introductory external resource can be found [http://www.ibm.com/developerworks/java/library/j-eaed13/index.html?ca=drs- here].
= Overview =
= Overview =

Richunger at 01:51, 10 June 2010 - 2010-06-10 01:51:11

←Older revision Revision as of 01:51, 10 June 2010
Line 138: Line 138:
VoiceXML interpreters can be compliant with both the 1.0 and 2.0 specifications by simply reading the root element, and then using different parsers depending on the value of the version attribute.
VoiceXML interpreters can be compliant with both the 1.0 and 2.0 specifications by simply reading the root element, and then using different parsers depending on the value of the version attribute.
-
In Apex, we do this a bit differently. Because all Apex classes are stored in our database, we simply have another column in the ApexClass table for the version. When users create a new class, this column defaults to the latest version number, but it is user-editable, in case they want to "upgrade" their class later to take advantage of new features.
+
I once worked on a NetBeans Platform project where the data files were of a particular XML schema with this type of version attribute. With each version, we tracked the changes we'd made to the file format (additions and modifications), and we'd write an XSLT script that could upgrade each version to the next one. So we had a set of files like:
 +
from1.xsl
 +
from2.xsl
 +
from3.xsl
 +
...
 +
Whenever my module opened a user's data file, it would check the version. If the version was not up to date, the XSLT scripts would be invoked sequentially, behind the scenes, to bring the data file up to date.
 +
 
 +
In [[Apex]], we do this a bit differently. Because all [[Apex]] classes are stored in our database, we simply have another column in the ApexClass table for the version. When users create a new class, this column defaults to the latest version number, but it is user-editable, in case they want to "upgrade" their class later to take advantage of new features.
One example of a change that is versioned in this manner is the way Apex handles floating point literals. In a class with version 17.0, the literal
One example of a change that is versioned in this manner is the way Apex handles floating point literals. In a class with version 17.0, the literal

Richunger: /* Evolution (Versioning, Deprecation) */ - 2010-06-10 01:45:36

Evolution (Versioning, Deprecation)

←Older revision Revision as of 01:45, 10 June 2010
Line 126: Line 126:
=== [[Evolution]] (Versioning, Deprecation)===
=== [[Evolution]] (Versioning, Deprecation)===
-
There are strict rules for evolution of [[Java]] [[API]]. The [[Chapter 6]] touches on this topic slightly, moving methods up and down the class hierarchy, adding methods, etc. When doing library design, you need to adjust to the [[Java]] (or other framework) rules. When writing a [[DSL]] you are basically creating your own rules, aren't you Rich?
+
Jarda: There are strict rules for evolution of [[Java]] [[API]]. The [[Chapter 6]] touches on this topic slightly, moving methods up and down the class hierarchy, adding methods, etc. When doing library design, you need to adjust to the [[Java]] (or other framework) rules. When writing a [[DSL]] you are basically creating your own rules, aren't you Rich?
It took me few years to understand the [[Java]] rules, how long it takes to specify own? There are tools which help you discover backward incompatibilities in [[Java]] like [[Sigtest]]. This all needs to be written from a scratch for [[DSL]] (btw. we would need something like that for [[Ant]] - as [[Ant]] [[XML]] build scripts supports overrides, inheritance, etc. - nothing like that is available, nobody will likely bother to write anything like that and rather we will rely on ''visual inspection'' and early testers to report bugs).
It took me few years to understand the [[Java]] rules, how long it takes to specify own? There are tools which help you discover backward incompatibilities in [[Java]] like [[Sigtest]]. This all needs to be written from a scratch for [[DSL]] (btw. we would need something like that for [[Ant]] - as [[Ant]] [[XML]] build scripts supports overrides, inheritance, etc. - nothing like that is available, nobody will likely bother to write anything like that and rather we will rely on ''visual inspection'' and early testers to report bugs).
-
TBD...
+
Rich: The great thing about DSLs is that you can completely change the syntax and semantics from version to version without breaking backwards compatibility. This is because you have complete control over the parser.
 +
 
 +
The clearest example of this is in certain XML derivatives. In VoiceXML, for example, the root element has a version attribute:
 +
 
 +
<vxml version="2.0">
 +
 
 +
VoiceXML interpreters can be compliant with both the 1.0 and 2.0 specifications by simply reading the root element, and then using different parsers depending on the value of the version attribute.
 +
 
 +
In Apex, we do this a bit differently. Because all Apex classes are stored in our database, we simply have another column in the ApexClass table for the version. When users create a new class, this column defaults to the latest version number, but it is user-editable, in case they want to "upgrade" their class later to take advantage of new features.
 +
 
 +
One example of a change that is versioned in this manner is the way Apex handles floating point literals. In a class with version 17.0, the literal
 +
12.4
 +
is a double. If you change the version of the class to 18.0, the same literal is a BigDecimal. If you want to specify a double literal, you'd type
 +
12.4d
 +
 
 +
When implementing this change in behavior, we basically add an if statement to our abstract syntax tree like:
 +
Object value;
 +
if (currentVersion > 17.0)
 +
value = new BigDecimal(parsedStringValue);
 +
else
 +
value = Double.valueOf(parsedStringValue);
 +
 
 +
DSL versioning is a very powerful concept. Unlike in a Java API, we can completely remove concepts from the latest version of the language instead of deprecating them, keeping the semantics of the language clear and simple. We do this because the old parser logic is still there, and old code will always use the old logic.

Richunger: /* Accessing Database */ - 2010-06-10 01:22:49

Accessing Database

←Older revision Revision as of 01:22, 10 June 2010
Line 99: Line 99:
==== Accessing Database ====
==== Accessing Database ====
-
Is it easier in [[DSL]] to access database structure, than in [[Java]]? I would expect so, I would expect that [[DSL]] can make it easy easy as [[Ruby]] on Rails or Grails - e.g. you just access object representing the database and all columns are accessible as fields. As soon as you change the database schema, new field will immediately be visible. This could be hard to simulate when writing [[Java]] libraries, or not?
+
Jarda: Is it easier in [[DSL]] to access database structure, than in [[Java]]? I would expect so, I would expect that [[DSL]] can make it easy easy as [[Ruby]] on Rails or Grails - e.g. you just access object representing the database and all columns are accessible as fields. As soon as you change the database schema, new field will immediately be visible. This could be hard to simulate when writing [[Java]] libraries, or not?
 +
 
 +
Rich: I would agree with this, but more because of the "Domain" part of "Domain Specific Language" than the "Language" part. The Apex runtime is aware of the database that is available, already has access to a connection pool, and already knows the tables and columns that have been configured by the organization. A library could take advantage of the same information, but it is really runtime information, and so would rely on a custom, domain-specific runtime. Once you have a general purpose language running in a domain specific runtime, you're really 95% of the way to what we did with Apex, which is to say "as long as we've got our own runtime, let's make things a bit easier on our customers by adding some keywords for the really common stuff". That starts you down the road of a full DSL.
===Learning Curve===
===Learning Curve===