Debugger

From APIDesign

Revision as of 19:42, 15 August 2013

Debugger is a program that helps to find out what other programs are doing. Often people use debugger to understand why their own code does not do, what they would like it to do. However debugger is incredibly useful for other purposes as well. Here is its story.

Building Modular systems

Chapter 1 of TheAPIBook discusses the art of building modern software systems and concludes that these days we are building systems that we don't understand. As an example it mentions that in order to write a dynamically generated web page one needs to understand HTML and possibly a Servlet API, but you don't need to have knowledge of anything that lays below: Glassfish server, Java VM, Unix libraries, Linux kernel, the VirtualBox nor even the hardware. At the end you can deploy a massive solution and in fact understand roughly one percent of it!

Some people may find it freightening that we use systems without real understanding, but The Art of Building Modern Software chapter concludes that such level of cluelessness is OK and in fact completely necessary. Brain has limited capacity, and the learning time is finite as well. As such we just can't understand everything. It is enough to know just the surface:

• the Servlet API - but not its actual implementation in the Glassfish server
• the user or command line interface of VirtualBox - to install Linux into it and to launch it
• the apt CLI - to install Java and Glassfish

Knowing these interfaces is enough. The details behind them are unimportant (if things go well).

Btw. this is also the reason why API design is important (command line parameters, as well as layout of files, are as important types of API as classes and their methods). Well designed application programming interfaces encourage cluelessness. The less knowledge people need to have to use a technology, the better the technology is. That is why it is still useful to read my Practical Design Book and learn how to design APIs properly! But let's now get back to topic of debugging.

Cluelessness of an API User

I am using word cluelessness for the blessed state when one does not need to know the internals behind APIs. However in fact I mean selective cluelessness: when there is no need to know to make things work, the less we know the better.

However when things go wrong, then we should concentrate on what is wrong and get as deep knowledge of the affected area as possible. We should select that we want to know everything about a particular part of (for example) Glassfish and learn every detail about it (for example why it throws some NullPointerException).

These days it should not be hard to increase our knowledge about our libraries. Most of them are open source and thus getting access to their source code is easy. Putting a breakpoint on appropriate line, launching own application under debugger is usually easy as well. Moreover it is also easy to play with the code - modify a line, compile (using Maven or make) and use the modified version. There are no barriers to increase our knowledge about Glassfish, Java, Linux, etc.

In spite of the above easiness, I don't see people doing it. I don't see them looking at sources of libraries they use. Rather than that they treat them as blackbox. In spite of all the options we have, some of us decided to stay clueless (in the negative meaning, without the important selective part).

The Googling Robot

Rather than trying to investigate the problem, the first reaction is to query google search machine. I have to admit, I do it too and the results from Stackoverflow web site and really valuable. It is selectively clueless to check whether somebody else tried to solve similar problem, so far this attitude is good.

However the really problematic state occurs as soon as the query yields no results. I've seen many situations when desperate programmers frustrated by unsatisfying search give up! Such googling only robots will however face inevitable destiny. These days google can recognize voice, drive a car - everything done by a brute force searches for correlations. The day when google will be able to create artificial intelligence that will program using searches is not far away. At that moment programmers will either choose to go out of their previous business or will have to become selectively clueless again - and learn how to use debugger.

Knowing Every Detail

Once a customer of the NetBeans Platform invited me for a consultation. It is not common to send the creator of a framework for a consultation, the usual strategy is to send the junior guy to find out if they can spell Java. Later send somebody more experienced and only if every effort fails, ask the guru. But even for me, as a creator of the NetBeans Platform, it is important to see real customer from time to time, so I took the opportunity to visit such important customer.

Certainly I can talk about architecture decisions and explain how to use NetBeans APIs on a higher level, but when it comes to details, I may often be in total cluelessness situation (there is a lot of NetBeans APIs check at http://bits.netbeans.org/7.3/javadoc/ and we design them as a team). As a result my most common answer to customer questions was: I don't know. Let's put a breakpoint somewhere!

Most of the time of my visit, we spend debugging. We located misbehaving API source code, placed a breakpoint somewhere and run the whole application. After few steps over we usually know what is the problem and could either workaround it, or (as I am not afraid to enhance API compatibly) modify the NetBeans source code to better suite customer needs. At the end the customer was satisfied - but was it really necessary for me to be there? Could not the guys use the debugger themselves?

When my visit was over I ironically commented that I did nothing else than debugging and patching NetBeans. At least I was useful to have the rights to make sure the patches get included in next version, I concluded. The customer's reply was shocking: No, you are more valuable for knowing where to place a breakpoint!.

Does the ability to place a breakpoint becomes the most important skill for a modern developer?

When Things Don't Happen

Debuggers were designed to allow developers to step through their own code. Certainly, when such feature hit market for the first time, it was a huge step forward. Ability to understand the behavior of own program by stepping through line by line and seeing the state of memory and variables is certainly useful. Much easier than running the whole program and then trying to guess on which line something went wrong.

However the proliferation of open source libraries and the fact that these days we rather assemble applications than write them makes us use debugger on foreign code rather than our own. Moreover we are no longer using just simple libraries (where we just made calls to them and continued our execution when the call returned), we are using frameworks. As a result it is the framework that makes calls to us depending on some declarative registration. Debugging own code is still the same, however something one needs to find out why something did not happen - why the framework decided not to call us. Are our debuggers good enough to help us resolve such problems?

Not really. Debuggers were originally designed to help us fix our own code when it misbehaved. Using them to find out why things that don't happen is not easy.

Side by Side Debugging

Few years ago I was working on a project to bring JDeveloper and NetBeans closer to each other. We started by replacing JDeveloper's module system based on Equinox by Netbinox (a mixture of Equinox and NetBeans that was the fastest OSGi container on the planet at that time). You can imagine replacing something as fundamental as base module system involves a lot of debugging.

The best approach was to run both systems side by side and watch for there differences. For a certain period of time we had a possibility to run the JDeveloper either on top of Equinox or on top of Netbinox - just by passing in different system property. When something went wrong under Netbinox, I performed the same operation in Equinox, observed the behavior by stepping through the affected method line by line and then compared the behavior with Netbinox. Usually it was quite obvious where the difference is and there were few.

This kind of side by side debugging is useful for finding out what is wrong when things don't happen. Because we had the old Equinox version which supposedly worked fine you can put breakpoint to the place that should be executed, when the breakpoint is hit in the old version, just remember the stack. The problem in the new Netbinox version has to be along the stack. Just put the breakpoint up to the methods higher on the stack and at one moment, it will be hit even in Netbinox. Once that happen you can again do line by line debugging and find out where the difference between Equinox and Netbinox is.

Side by side debugging is not useful only when we are rewriting the one system to new version. Even if something is wrong when we are using a framework, we can usually find a tutorial or demo application that is supposed to do the same as we are trying to achieve. At that moment, we can compare the behavior of working demo, with misbehavior of our application. Side by side debugging is useful for anybody.

Insufficient Documentation of Maven Tasks

TBD: taking maven to extreme

Debugging without Source Code

TBD: iOS

Use the Debugger. Stupid!

TBD