Just recently I tried rewriting OpenGL’s routines that handle matrix calculations as part of a lecture at the university. Pretty soon I came to the conclusion, that I had to optimize my code if I wanted to compete with the implementation provided by OpenGL.

This post presents at quick look at CPU Sampler which helped me making well-founded decisions resulting in a faster implementation.

Howto

Once you’ve developed your software with Xcode and want to profile it using CPU Sampler, all you have to do is to click on Run – Run with Performance Tool – CPU Sampler. This will start the tool and run your application; you can start and stop the recording as you see fit.

As soon as you end your application you can have a look at the samples collected during the last run. This is the interesting part because you can review how long a certain function was running and the percentage tells you how long this took compared to the overall run.

Try playing around with the various options. In most situations you can stick with the defaults but there’re more interesting things to look at if you want to. For example try changing the Active thread from All threads to something else and the view will show the corresponding samples only.

Example

As I said, I tried rewriting OpenGL’s routines that handle matrix calculations. This was meant as an exercise to understand the operations needed that changed the matrices. Of course, the implementation was way slower than OpenGL and I should try speeding it up.

Using CPU Sampler I immediately identified a bottleneck and a major difference between my implementation and the one that comes with OpenGL. The slow part of my code could be found in repeated calls to cos and sin. Since OpenGL seems to work with the libraries of my graphics card directly I guess it tries to do certain computations there.

What I now did was substituting the calls to cos and sin with code that used a lookup table to retrieve precalculated values. This resulted in a major performance boost – still, OpenGL’s code is twice as fast as my one.

Have a look at the following screenshots. They show a run using OpenGL’s implementation of matrix manipulations, my own and an optimized version of my own implementation. Comparing the naive and optimized version of my code clearly shows that the optimized variant doesn’t spent much time calculating cos and sin.

Conclusion

I wanted to show that using a profiling tool like CPU Sampler that comes with Apple’s Xcode may make your life much easier if you’re trying to optimize your application. This helps you to make wise decisions when it comes to optimizing code so you don’t have to obfuscate your software with hard to read code blocks that have a very little effect regarding performance.

It’s nice to see that Apple ships tools like this with its IDE. Have a look at the other performance tools and you’ll be amazed how many cools things Xcode is capable of – things you wouldn’t even dream about in other IDE’s.

• using an Iterable
• toCharArray()
• charAt()
• calling charAt() on a CharSequence

You can download the Eclipse project as a tar or zip or browse the code online here; have a look at the StringIteratorTest class first.

Results

Without further ado here are the results:

Conclusion

If speed is what you want you shouldn’t use Iterators but one of the other solutions instead. On the other hand if you just like to play with Iterators and classes that implement Iterable you may want to choose the slower solution.

• Commons BeanUtils
• Spring BeanUtils
• Dozer

Furthermore I used my own implementation and an implementation that’s hardcoded by hand.

The Eclipse project with the sample code for this post can be downloaded as tar.gz or zip or can be viewed online here.

Testing

The performance test creates two beans: one is filled and the other gets filled; this is done over and over again in a big for loop. I coded a by hand implementation as a reference to check whether the process of creating all these new objects is slow. This implementation copies the properties from one bean to the other like so:

bean2.setFoo(bean1.getFoo());
bean2.setBar(bean1.getBar());
bean2.setTee(bean1.getTee());

All the other frameworks had to do the same thing. Have a look at the results (note that I set the y-axis to logarithmic scale):

Obviously there’s no problem with all the created objects: the by hand implementation just consists of the created objects and some method calls to copy the properties. This seems to be pretty fast compared to the other solutions.

Let us have a look at a comparison between the by hand implementation and the frameworks when they have to fill 100k objects:

For example: the by hand implementation beats Dozer by a factor of 970, i.e. it’s 970 times faster.

Conclusion

The results can be explained with the different richness of features. My Introspector and the implementation from the Springframework have very few features and are pretty fast. The Commons BeanUtils implementation has a lot of features (e.g. all this DynaBean stuff). And finally Dozer: I haven’t looked at the code yet, but it seems to be very powerful.

So once more it’s a trade-off between speed and features. Obviously you can’t have both so decide which one your application needs.

If you’d like to check out the sample code that accompanies this post, you can download the Eclipse project as tar.gz or zip; make sure to install the AspectJ Development Tools. You can browse the code online here.

How to profile methods

Writing an aspect that adds advice to the methods you’d like to profile is as easy as this. You define a pointcut that matches the methods you’d like to monitor and define an around advice that calculates the time it took the method to execute. Have a look at the following piece of code:

@Around("profile()")
public void aroundInterestingMethods(ProceedingJoinPoint thisJoinPoint)
            throws Throwable {
  final long start, end;
  start = System.nanoTime();
  thisJoinPoint.proceed();
  end = System.nanoTime();
  // calculate time: (end - start) / 1000) / 1000;
}

All that’s left to do is to define a pointcut named profile, that matches the methods you’d like to monitor. Since the pointcut will be variable, I wrote an abstract aspect that can be subclassed for different monitoring needs.

In a subclass you would implement the profile method like so, if you’d like to monitor all methods named run:

@Pointcut("execution(* *.*.run())")
protected void profile() {}

What about the results?

If you would run the sample code, you could see this:

    Results for Runner.run (1002ms)
    Method: TaskB.run() took 600ms (60.0%)
    Method: TaskA.run() took 400ms (40.0%)

Once the run method of the Runner class comes to an end, we can see these statistics, produced by the afterTopLevelMethod method from the profiling aspect.

Based on this textual results it would be pretty easy to produce a graphical representation with gnuplot or JFreeChart. I could imagine a picture like this:

Although this looks very simple, a picture might be easier to absorb if there would be more methods.

Conclusion

As we’ve seen, profiling with AspectJ can be implemented straight forward. My simple example showed how to monitor certain methods and calculate the their execution time. The authors of this paper point us to some more things we might want to profile, e.g. heap space. Although I haven’t tried it yet, it should be possible to implement this too.

Suppose you’ve built an application that uses standard JavaBeans to represent business objects of your problem domain. Then you decided to expose your application as a web service that returns XML: you created a XML Schema for JAXB and generated a second model with it. Now all you would have to do is to fill the generated model and use JAXB to read/write it to your web service.

Since you’re a hacker and avoid repetitive work like the plague you write a tool that introspects your original beans and populates their properties to the generated JAXB beans and vice versa. This way you don’t have to write lots and lots of code that looks like:

// for marshalling
jaxbBean.setFoo(bean.getFoo());
// for unmarshalling
bean.setFoo(jaxbBean.getFoo());

You might ask why you don’t use the generated model in the first place and get rid of the old one. First of all this would be less fun and as a serious answer you probably don’t want to couple your application too tight to JAXB, because you might want to switch to another Java-to-XML binding framework (e.g. Castor) in the future.

BeanIntrospector

I wrote a small class (88 LOC) that’ll copy properties from one bean to another; it uses the aforementioned Introspector class. Although this is a pretty straight forward thing it introduces a major performance penalty.

The final class is now a product of several development cycles:

1. my first version was a brute force attempt with some for loops
2. the second version stored the methods of the JavaBeans in a lookup table; so it could look up the methods faster
3. in addition to the second version, the third version tries to make no duplicate calls to e.g. getClass() – these things are stored in variables now.

You can download the Eclipse project as tar.gz or zip file or browse the code here; make sure that you have the Maven2 plugin for Eclipse installed.

Tests

To evaluate the different versions, I coded a simple performance test that compares the performance of my BeanIntrospector implementation to code that does the same thing by hand, i.e. calling every getter/setter of two beans one after another.

The first chart shows a comparison of the different implementations.

This leaves no doubt that the boring solution is by far the fastest. Another thing you can notice is that tuning the code can help a lot: while the first implementation takes more than 100ms to fill 1000 JavaBeans, the tuned version takes about 40ms to do the same thing. In addition to that the final version is more readable than the brute force variant.

The second chart shows a comparison between the final implementation and doing the same thing by hand. Note that I set the y-axis to logarithmic scale.

Conclusion

If performance is important for your application you should explicitly copy the properties from one bean to another. If you’ve got beans with a lot of properties, write yourself a small tool that’ll generate the code for you. On the other hand if you need a fast hack you might want to use the BeanIntrospector class I wrote or write yourself a similar solution.

Some performance tests

Recently I read Code Complete, an excellent book by Steve McConnell, that has got some good ideas about testing performance. According to McConnell’s habit of measuring the code’s performance instead of telling anecdotes I wrote some tests that’ll:

• create an object and call some methods on it
  (JavaPersonBean and GroovyPersonBean)
• create a one- and two-dimensional array and fill it with some values
  (JavaArray and GroovyArray)
• compare two integers
  (JavaCompare and GroovyCompare)

You can download the Eclipse project as tar.gz or zip or browse the code here; make sure that you have the Groovy and Maven2 plugins installed.

Have a look at the following chart that shows the results:

There are no bars for some classes because they virtually took no time to execute. On the other hand some tests took too long so I cut them off, i.e. GroovyPersonBean and GroovyArray aren’t displayed in the chart after 10⁵ and 10³ test runs respectively.

I ran these tests on a 1,83 GHz Intel Core Duo and used Groovy 1.0 and Java 1.5.0_07 under MacOS X. The tests are everything but accurate and should just give us a rough idea of the performance of Groovy and Java.

What do we learn from these tests?

First off Groovy seems to be slow if you’re making just one test run. The explanation for this are static fields that need to be initialized the first time you instantiate a Groovy class. The GroovyPersonBean class is a good example of this phenomenon: there’s a peek at the beginning and the next time we get measurable results is at 10³ test runs.

If you take a look at the number of test runs, Java seems to be three powers of ten faster than Groovy, e.g. Groovy takes circa 2s at 10³ test runs while Java takes 2s at 10⁶ test runs (JavaArray and GroovyArray). This holds true for almost all test cases.

Improvements in readability? Productivity?

As a side note I thought about the readability of the code. I haven’t used Groovy in a production environment yet, but the things I’ve coded so far just look nice. The learning curve is steep if you’ve got a background in Java and I certainly saved some time writing Groovy code instead of writing the equivalent in Java. I guess that readability, productivity and motivation might go up with Groovy because it’s just fun using it instead of ye good ol’ verbose Java language.

But as I said: since I haven’t tested it in a real project yet, I eventually might have no clue about this.

Conclusion

Although it seems like Groovy is a lot slower compared to Java we have to reconsider the tests I ran. First, there’s the test that creates a bean and calls some setters on it. Even if you create 10⁵ of these groovy beans it’ll just take less than a half of a second. I think that Groovy isn’t losing on this one because who wants to create so many objects in the first place? So I would say that Groovy is well suited as a substitute for regular Java beans; the fact that Groovy generates getters and setters automatically makes it even better.

Second, I created an array and assigned values to it. I haven’t come up with some other tests yet to confirm the assumption that Groovy lacks performance for code that does something. If I have some time to spend I’ll investigate this further. The third pair of tests just compare two integers and because this stands separate from the other tests I don’t go into detail about it.

Coming to a conclusion I would say that you can use Groovy to code beans without imposing a performance penalty. This might not be true for real groovy code, but this has to be investigated further.