Virtualization

From APIDesign

Revision as of 15:46, 14 June 2010

Virtualization is an important aspect in cluelessness. Instead of improving own application, or making it more secure, one can run it in an isolated, dedicated virtual computer. It may require more memory or more powerful processors, but the need of software quality is lower and thus one can achive more by knowing less - the credo of cluelessness.

OpenJDK

I wanted to run this site's associated hudson builder in an isolated environment. Originally I was using vserver, but after my hard disk crashed, I decided to try VirtualBox.

I am running Kubuntu as host and also a stripped down version of Kubuntu as a guest. I have given the guest 384MB of memory and 400MB of swap. To my biggest surprise, running even smallest Java program on OpenJDK in such environment allocated 650MB(!!!) of virtual addressing space. Just check the top output:

Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.7%us,  4.7%sy,  0.0%ni, 94.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    379872k total,   306828k used,    73044k free,    40164k buffers
Swap:   407544k total,    16064k used,   391480k free,   111272k cached
 
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                     
 4129 hudson    20   0  655m  10m 7484 S  2.3  2.8   0:01.02 java.openjdk
 4210 hudson    20   0  5248 3500 1556 S  0.0  0.9   0:00.87 bash                                                                                                        
 4144 root      20   0  8856 3048 2412 S  0.0  0.8   0:00.40 sshd                                                                                                        
 1619 postfix   20   0  6244 2544 1812 S  0.0  0.7   0:00.12 tlsmgr                                                                                                      
  951 hudson    20   0  5248 2008 1280 S  0.0  0.5   0:01.08 bash

650MB of virtual space is a lot. If you moreover realize that hudson starts multiple Java VMs in parallel, it is easy to see that the system can't work. Indeed it did not: swap was full all the time, sometimes the VirtualBox machine rebooted, etc.

For a while I was searching for a solution. I tried to pass in -mx64M into every started process, but as far as I can tell, this is not easy. There does not seem to be any configuration file the java launcher reads (jvm.cfg is not the one, one cannot specify there default command line arguments).

One can specify the parameter on command line, which is not practical. If you don't start the process yourself, how do you influence command line? That is why I renamed java to java.openjdk and replaced java with a shell script to call the java.openjdk with proper parameter. This worked, but only for java, not for javadoc, etc. So I was not satisfied with the solution at all.

Then a friend of mine suggested to try official JDK, not just OpenJDK variant. I hesitated for a while, but then I decided to try it. Here is the top output:

Tasks:  81 total,   2 running,  79 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.0%us,  2.3%sy,  0.0%ni, 96.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    379872k total,   307924k used,    71948k free,    40388k buffers
Swap:   407544k total,    16056k used,   391488k free,   111424k cached
 
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                     
 4306 hudson    20   0  239m  10m 6956 S  0.0  2.8   0:00.48 java                                                                                                        
 4210 hudson    20   0  5248 3500 1556 S  0.0  0.9   0:00.88 bash                                                                                                        
 4144 root      20   0  8856 3048 2412 S  0.0  0.8   0:00.40 sshd

Better. The 240M of occupried space is still much more than I'd like to see (no wonder nobody offers free hosting of Java applications), but at least it fits into real memory. By using JDK 1.6.0_20 I can really run my Hudson in VirtualBox with 384M of memory.

Conclusion? OpenJDK seems to contain a bug in a launcher which makes it completely unsuitable for running in a virtualized environment! What a shame for a standard Java implementation present in each Linux distribution!

Too Big Initial Limit

The difference between OpenJDK and JDK seems to be result of this change. It increases the heap limit to 512MB and the perm gen size to 128MB claiming that if Eclipse needs it, every other Java application will need it as well.

No it won't. If you run multiple VMs with for example JUnit tests in parallel, the performance will suffer.

It would indeed be great if the VM internal structures could start small and grow, but if that is not possible for some reasons now, then the ergonomics need to prevent allocating more memory than is physically available. Blindly increasing the default is contra productive.

<comments/>