Kenneth Kousen's complete blog can be found at: http://kousenit.wordpress.com
Friday, January 13, 2012
I love teaching Groovy to existing Java developers, because they have such a hard time holding back Tears Of Joy when they see how much easier life can be. Today, though, I did a quick demo that resulted in a line of Groovy that was so amusing I had to post it here.
Consider a trivial POGO (Plain Old Groovy Object) called Course:
class Course
String name
int days
String toString() { "($name,$days)" }
}
The goal was to take a collection of courses and sort it by the number of days. That’s really easy in Groovy:
def courses = [
new Course(name:'Groovy',days:4),
new Course(name:'Grails',days:3),
new Course(name:'Spring',days:4),
new Course(name:'Hibernate',days:3)
]
assert courses.toString() == '[(Groovy,4), (Grails,3), (Spring,4), (Hibernate,3)]'
courses.sort { it.days }
assert courses*.days == [3, 3, 4, 4]
The
sort method in the java.util.Collection class is part of the Groovy JDK, meaning it’s one of the methods Groovy adds to the standard Java libraries. It takes a closure of either one or two arguments. In this case, I’m using the one-argument closure, which is used to select a property on which to base the sort. By specifying it.days in the closure, I’m telling the sort method to sort the courses based on their days property. Then I verify that the sort worked by checking that the courses are the right order, using the spread-dot operator to just look at the number of days.
In class the question that always comes up is, can I sort by days and then by name? In other words, if two courses have the same number of days, can I then sort by the name property?
That’s what the two-argument closure on the sort method is for. The two arguments are references to any pair of courses, and the closure should return a negative number, zero, or positive number according to whether the first course is less than, equal to, or greater than the second.
Here’s where things get amusing. The sort I want is:
courses.sort { a,b ->
a.days <=> b.days ?: a.name <=> b.name
}
assert courses.toString() == '[(Grails,3), (Hibernate,3), (Groovy,4), (Spring,4)]'
The body of the closure on
sort uses the spaceship operator <=>, which returns -1, 0, or 1 depending on whether the left side is less than, equal to, or greater than the right side. I use spaceship to compare the days properties. Then I add the Elvis operator ?: which means if the days comparison is not zero, use it, but otherwise use the following comparison, which uses another spaceship to compare by name.
It’s only after writing the code in class that one of the students pointed out that I had Elvis in between two spaceships, leading to the following observations:
- The spaceships are there to return Elvis to his home planet
- It takes two spaceships, working in tandem, to carry Elvis away, in much the same way two swallows can carry a coconut in tandem
- Therefore, the Elvis being carried away must be the fat Elvis from the 70s, rather than the thin, cool Elvis from the 50s
Either way, after the sort is finished, Elvis has left the building.
Thank you, thank you very much.
Monday, January 2, 2012
I finished revising the testing chapter in Making Java Groovy (the MEAP should be updated this week), but before I leave it entirely, I want to mention a Groovy capability that is both cool and easy to use. Cool isn’t the right word, actually. I have to say that even after years of working with Groovy, what I’m about to describe still feels like magic.
Here’s the issue: I have a class that uses one of Google’s web services, and I want to test my class even when I’m not online. That means I need to mock the dependency, which isn’t all that hard. The problem is that there’s no explicit way to get my mock object into my own service. Yet, with Groovy’s MockFor and StubFor classes, I can mock a dependency, even when it’s instantiated as a local variable inside my class.
Let me show you the code. I’ll start with a simple POGO called Stadium:
class Stadium {
String street
String city
String state
double latitude
double longitude
String toString() {
"($street,$city,$state,$latitude,$longitude)"
}
}
I use this in my Groovy Baseball application, which accesses MLB box scores online and displays the daily results on a Google Map. The
Stadium class holds location data for an individual baseball stadium. When I use it, I set the street, city, and state and have the service compute latitude and longitude for me.
My Geocoder class is based on Google’s restful geocoding service.
class Geocoder {
String base = 'http://maps.google.com/maps/api/geocode/xml?'
void fillInLatLng(Stadium stadium) {
String urlEncodedAddress =
[stadium.street, stadium.city, stadium.state].collect {
URLEncoder.encode(it,'UTF-8')
}.join(',+')
String url = base + [sensor:false, address:urlEncodedAddress].collect { it }.join('&')
def response = new XmlSlurper().parse(url)
String latitude = response.result.geometry.location.lat[0] ?: "0.0"
String longitude = response.result.geometry.location.lng[0] ?: "0.0"
stadium.latitude = latitude.toDouble()
stadium.longitude = longitude.toDouble()
}
}
First I take the stadium’s street, city, and state and add them to a list. Then the
collect method is used to apply a closure to each element of the list, returning the transformed list. The closure runs each value through Java’s URLEncoder. In looking at the Google geocoder example, I see that they separate the encoded street from the city and the city from the state using “,+“, so I do the same using the join method.
The Google geocoding service requires a parameter called sensor, which is true if the request is coming from a GPS-enabled device and false otherwise. In the Groovy map, I set its value to false, and set the value of the address parameter to the string from the previous line. The collect method on the map converts each entry to “key=value“, so joining with an ampersand and appending to the base value gives me the complete URL for the stadium.
Most restful web services try to provide their data in a format requested by the user. The content negotiation is usually done through an “Accept” header in the HTTP request, but in this case Google does something different. They support only XML and JSON output data, and let the user select which one they want through separate URLs. The base URL in the service above ends in xml. Google lists the JSON version as preferred, but that’s no doubt because they expect the requests to come through their own JavaScript API. I’m making the request using Groovy, and the XmlSlurper class makes parsing the result trivial.
(Since Groovy 1.8, the JsonSlurper class also makes parsing JSON trivial, and I have a version that does that, too, but the difference really isn’t significant here.)
The resulting block of XML that is returned by the service is fairly elaborate, but as you can see from my code, the data is nested through the elements GeocodeResponse/result/geometry/location/lat and GeocodeResponse/result/geometry/location/lng. After invoking the parse method (which returns the root element, GeocodeResponse), I just walk the tree to get the values I want.
I do have to protect myself somewhat. The Google service isn’t nearly as deterministic as I would have expected. Sometimes I get multiple locations when I search, so I added the zero index to make sure I always get the first one. Also, some time during the last year Google decided to start throttling their service. I have a script that uses the Geocoder for all 30 MLB stadiums, and unfortunately it runs too fast (!) for the Google limits. I know this because I start getting back null results if I don’t artificially introduce a delay. That’s why I put in the Elvis operator with the value 0.0 if I don’t get a good answer.
So much for the service; now I need a test. If I know I’m going to be online, I can write a simple integration test as follows:
import static org.junit.Assert.*;
import org.junit.Test;
class GeocoderIntegrationTest {
Geocoder geocoder = new Geocoder()
@Test
public void testFillInLatLng() {
Stadium google = new Stadium(street:'1600 Ampitheatre Parkway',
city:'Mountain View',state:'CA')
geocoder.fillInLatLng(google)
assertEquals(37.422, google.latitude, 0.01)
assertEquals(-122.083, google.longitude, 0.01)
}
}
I’m using Google headquarters, because that’s the example in the documentation. I invoke my Geocoder’s
fillInLatLng method and check the results within a hundredth of a degree.
(The fact that the Google geocoder returns answers to seven decimal places is evidence that their developers have a sense of humor.)
Now, finally, after all that introduction, I reach the subject of this post. What happens if I’m not online? More to the point, how do I test the business logic in the fillInLatLng method without requiring access to Google?
What I need is a mock object, or, more precisely, a stub.
(A stub stands in for the external dependency, called a collaborator. A mock does the same, but also validates that the caller accesses the collaborator’s methods the right number of times in the right order. A stub helps test the caller, and a mock tests the interaction of the caller with the collaborator, sometimes called the protocol. Insert obligatory link to Martin Fowler’s “Mocks Aren’t Stubs” post here.)
In my Geocoder class, the Google service is accessed through the parse method of the XmlSlurper. Even worse (from a testing point of view), the slurper is instantiated as a local variable inside my fillInLatLng method. There’s no way to isolate the dependency and set it from outside, either as an argument to the method, a setter in the class, a constructor argument, or whatever.
That’s where Groovy’s StubFor (or MockFor) class comes in. Let me show it in action. First, I’ll set up the answer I want.
String xml = '''
<root><result><geometry>
<location>
<lat>37.422</lat>
<lng>-122.083</lng>
</location>
</geometry></result></root>'''
def correctRoot = new XmlSlurper().parseText(xml)
The xml variable has the right answer in it, nested appropriately. I use the
XmlSlurper to parse the text and return the root of the tree I want.
Now I need a Stadium to update. I deliberately put in the wrong street, city, and state to make sure that I only get the right latitude and longitude if I insert the stub correctly.
Stadium wrongStadium = new Stadium(
street:'1313 Mockingbird Lane',
city:'Mockingbird Heights',state:'CA')
(Yes, that’s a pretty obscure reference. For details, see here. And yes, I’m old. If you’re in the right age group, though, you now have the show’s theme song going through your head. Sorry.)
The next step is to create the stub and set the expectations.
def stub = new StubFor(XmlSlurper)
stub.demand.parse { correctRoot }
The
StubFor constructor takes a class as an argument and builds a stub around it. Then I use the demand property to say that when the parse method is called, my previously computed root is returned rather than actually going over the web and parsing anything.
To put the stub into play, invoke its use method, which takes a closure:
stub.use {
geocoder.fillInLatLng(wrongStadium)
}
That’s all there is to it. By the magic of Groovy metaprogramming, when I invoke the
parse method of the XmlSlurper — even when it’s instantiated as a local variable — inside the use block the stub steps in and returns the expected value.
For completeness, here’s the complete test class:
import static org.junit.Assert.*
import groovy.mock.interceptor.StubFor
import org.junit.Test
class GeocoderUnitTest {
Geocoder geocoder = new Geocoder()
@Test
public void testFillInLatLng() {
Stadium wrongStadium = new Stadium(
street:'1313 Mockingbird Lane',
city:'Mockingbird Heights',state:'CA')
String xml = '''
<root><result><geometry>
<location>
<lat>37.422</lat>
<lng>-122.083</lng>
</location>
</geometry></result></root>'''
def correctRoot = new XmlSlurper().parseText(xml)
def stub = new StubFor(XmlSlurper)
stub.demand.parse { correctRoot }
stub.use {
geocoder.fillInLatLng(wrongStadium)
}
assertEquals(37.422, wrongStadium.latitude, 0.01)
assertEquals(-122.083, wrongStadium.longitude, 0.01)
}
}
Since I’m only calling one method and only calling that method once, a stub is perfectly fine in this case. I could invoke the
verify method if I wanted to (MockFor invokes verify automatically but StubFor doesn’t), but it doesn’t really add anything here.
There is one limitation to this technique, which only comes up when you’re doing the sort of Groovy/Java integration that is the subject of my book. You can only use StubFor or MockFor on a class written in Groovy.
All this code is available in the book’s Github repository at http://github.com/kousen/Making-Java-Groovy. This particular example is part of Chapter 5: Testing Java and Groovy.
As a final note, I should say that I dug into all of this, worked out all the details, and tried it in several examples. Then I finally did what I should have done all along, which was look it up in Groovy In Action. Of course, there it was laid out in detail over about five pages. Someday that will stop happening to me.
Monday, December 12, 2011
Recently I saw a post by someone (I think it was @jbarnette, but it was retweeted to me) suggesting that there should be some alternate log levels, like fyi, omg, or even wtf. I thought that was pretty funny, but then it occurred to me I could probably implement them using Groovy metaprogramming.
As a first attempt, consider the following simple example that adds the fyi and omg methods to java.util.logging.Logger:
import java.util.logging.Logger
Logger.metaClass.fyi = { msg -> delegate.info msg }
Logger.metaClass.omg = { msg -> delegate.severe msg }
For those who haven’t used Groovy much, the
metaClass property is associated with every class in Groovy, and allows you to add methods and properties to the class. Here the fyi method is defined by assigning it to a one-argument closure whose implementation is to invoke the (existing) info method in Logger, with the msg argument. Likewise, omg is assigned to the severe method. Therefore, an invocation like:Logger log = Logger.getLogger(this.class.name) log.fyi 'for your information' log.omg 'oh my goodness'
results in
Dec 12, 2011 10:09:02 PM java_util_logging_Logger$info call
INFO: for your information
Dec 12, 2011 10:09:02 PM java_util_logging_Logger$severe call
SEVERE: oh my goodness
The methods work, but the output isn’t really what I want. The messages get passed through, but the output shows INFO and SEVERE rather than FYI and OMG.
It turns out it takes a bit of work to define a custom log level. Levels are defined using the java.util.logging.Level class, which predefines levels like Level.INFO, Level.WARNING, and Level.SEVERE. The Level class has a protected constructor which can be used to make new levels. I therefore adding a class called CustomLevel, as follows:
import java.util.logging.Level
class CustomLevel extends Level {
CustomLevel(String name, int val) {
super(name,val)
}
}
Each level gets an integer value. On my Windows 7 system (sorry) using JDK 1.6, the actual values of some of the defined levels are:
import java.util.logging.Level
println "$Level.INFO: ${Level.INFO.intValue()}"
println "$Level.WARNING: ${Level.WARNING.intValue()}"
println "$Level.SEVERE: ${Level.SEVERE.intValue()}"
INFO: 800
WARNING: 900
SEVERE: 1000
My second attempt was then to define a Groovy category, so that I could replace a couple of the existing levels in a controlled fashion.
import java.util.logging.Level
import java.util.logging.Logger
class SlangCategory {
static String fyi(Logger self, String msg) {
return self.log(new CustomLevel('FYI',Level.INFO.intValue()),msg)
}
static String lol(Logger self, String msg) {
return self.log(new CustomLevel('LOL',Level.WARNING.intValue()),msg)
}
}
Logger log = Logger.getLogger(this.class.name)
use(SlangCategory) {
log.fyi 'this seems okay'
log.lol('snicker')
}
Each of the logging methods in the
Logger class (like info() or warning()) delegate to the log() method, which takes two arguments — an instance of Level, and a message String. I therefore used the category to replace the INFO and WARNING levels with FYI and LOL. The output is now:
Dec 12, 2011 10:20:29 PM sun.reflect.NativeMethodAccessorImpl invoke0
FYI: this seems okay
Dec 12, 2011 10:20:29 PM sun.reflect.NativeMethodAccessorImpl invoke0
LOL: snicker
Once again, this is just replacing existing levels, though it does at least have the new level name in the output string.
To really do this right, though, I wanted to be able to define new levels arbitrarily without having to hardwire them. That meant overriding the methodMissing method in the metaClass, using what Jeff Brown describes as the “intercept, cache, invoke” pattern for metaprogramming. Here’s the result, which I’ll explain after the code.
import java.util.logging.*
Logger.metaClass.methodMissing = { String name, args ->
def impl = { Object... varArgs ->
int val = Level.WARNING.intValue() +
(Level.SEVERE.intValue() - Level.WARNING.intValue()) * Math.random()
def level = new CustomLevel(name.toUpperCase(),val)
delegate.log(level,varArgs[0])
}
Logger.metaClass."$name" = impl
impl(args)
}
Logger log = Logger.getLogger(this.class.name)
log.wtf 'no effin way'
log.whoa 'dude, seriously'
log.rofl "you're kidding, right?"
The
methodMissing method of the metaClass takes two arguments: the name of the method, and the arguments passed to it. Whenever you invoke a method that doesn’t exist, methodMissing gets invoked. That’s the “intercept” part.
The implementation is to define a closure that takes any number of arguments. Inside the closure, I computed a random value between Level.WARNING and Level.SEVERE, and then used that value to instantiate a custom level. The defined level and the new value were used in the log method on the closure’s delegate property (in this case, the logger) to log the message at the new level.
Finally, the "$name" method to the metaClass (which evaluates the name variable — otherwise the method added would just be called name) is assigned to the impl closure. That’s the “cache” part. Finally, the implementation is called, which is the “invoke” part.
Now I can use a log level with whatever name I want. The output of this script is
Dec 12, 2011 10:31:35 PM sun.reflect.NativeMethodAccessorImpl invoke0
WTF: no effin way
Dec 12, 2011 10:31:35 PM sun.reflect.NativeMethodAccessorImpl invoke0
WHOA: dude, seriously
Dec 12, 2011 10:31:35 PM sun.reflect.NativeMethodAccessorImpl invoke0
ROFL: you're kidding, right?
The demos are nice, but this really ought to be tested. I’m reasonably comfortable with this level (snicker) of metaprogramming, but I’d feel a lot better if I had a real test for it.
That took a fair amount of digging. It turns out that the inimitable Dierk Koenig (lead author of Groovy in Action, known as #regina on Twitter; second edition available now through the Manning Early Access Program) wrote a class called groovy.lang.GroovyLogTestCase. That class has a static method called stringLog. The GroovyDocs say:
“Execute the given Closure with the according level for the Logger that is qualified by the qualifier and return the log output as a String. Qualifiers are usually package or class names. Existing log level and handlers are restored after execution.”
It took me a while figure out how to use that, but eventually I got it to work. It automatically captures the console appender output, so the resulting test looks like this:
import java.util.logging.Level
import java.util.logging.Logger
class LoggingTests extends GroovyLogTestCase {
String baseDir = 'src/main/groovy/metaprogramming'
void testWithoutCustomLevel() {
def result = stringLog(Level.INFO, without_custom_levels.class.name) {
GroovyShell shell = new GroovyShell()
shell.evaluate(new File("$baseDir/without_custom_levels.groovy"))
}
assert result.contains('INFO: for your information')
assert result.contains('SEVERE: oh my goodness')
}
void testSlangCategory() {
def result = stringLog(Level.INFO, use_slang_category.class.name) {
GroovyShell shell = new GroovyShell()
shell.evaluate(new File("$baseDir/use_slang_category.groovy"))
}
assert result.contains('FYI: this seems okay')
assert result.contains('LOL: snicker')
}
void testEMC() {
def result = stringLog(Level.INFO, use_emc.class.name) {
GroovyShell shell = new GroovyShell()
shell.evaluate(new File("$baseDir/use_emc.groovy"))
}
assert result.contains('WTF: no effin way')
assert result.contains('WHOA: dude, seriously')
assert result.contains("ROFL: you're kidding, right?")
}
}
I should mention a couple of minor points. First, the class associated with a script has the same name as the file containing it, so the classes in the
stringLog method are the script names. Second, in case you were wondering, the emc part of the script name stands for ExpandoMetaClass.
I spent a very pleasant evening working on this, and learned a few things:
1. Groovy metaprogramming is fun,
2. Now I know how to use GroovyLogTestCase, which is not at all well documented, and
3. I’ll go to all sorts of trouble to avoid working on what I’m supposed to be working on, especially if it involves a joke.
I added all this to my book’s source code. What book, you ask? Why, Making Java Groovy, available through the Manning Early Access Program (MEAP) at http://manning.com/kousen. I don’t know, however, if I’ll have room in the text to include all this.
log.marketing('Please forgive the mandatory advertising for my book')
Tuesday, December 6, 2011
I try to keep up with developments in the Groovy and Grails worlds. I really do. I follow most of the core team members on Twitter. I listen to the Grails Podcast when I can. I go to many conferences and attend other talks when I’m not speaking. I try to follow the email lists, though they’re way too high volume. I even have a Google+ account, though I don’t check it very often.
It’s all too much, actually. As I get older, I find that keeping up isn’t just a question of time, it’s a question of energy. Sometimes I’ll manage to catch up on my Twitter feed and am too tired to do much else.
So new developments slip by me. I guess that’s good, since it’s indicative of an active ecosystem, but it can be frustrating at times.
The one that forms the subject of this blog post is pretty trivial and arguably not worth writing about, but I missed it when it first came out so I thought I’d document it here.
Say you have a script in Groovy and you want to test it. You can execute the script programmatically using GroovyShell, and supply any needed variables with an instance of the Binding class.
Here’s a trivial example. I have the following massively complex, powerful script:
z = x + y
Say this is saved in a file called ‘
math.groovy‘. Since none of the variables x, y, and z are declared at all in the script (not even using def), they can be accessed through a Binding object.
Binding binding = new Binding()
binding.x = 3; binding.y = 4
GroovyShell shell = new GroovyShell(binding)
shell.execute(new File('math.groovy'))
assert 7 == binding.z
This is easy enough to convert into a test case, as a subclass of
GroovyTestCase.
class ScriptTests extends GroovyTestCase {
void testMath() {
Binding binding = new Binding()
binding.x = 3; binding.y = 4
GroovyShell shell = new GroovyShell(binding)
shell.evaluate(new File('math.groovy'))
assertEquals 7, binding.z
}
}
By extending
GroovyTestCase, this entire script can be executed at the command line or inside the Groovy Console without adding any additional library dependencies of any kind (not even JUnit). When I ran it, the result was:
.
Time: 0.037
OK (1 test)
That’s all well and good. I was writing up examples like this for my book (Making Java Groovy, available through the Manning Early Access Program at http://manning.com/kousen) and decided to look at the Groovy API for
GroovyTestCase. Lo and behold, what do I stumble upon but a class called GroovyShellTestCase. The class was authored by Alex Tkachman (so you know it’s good *Which is what I like to call the CAP Design Pattern — how to take an error in one small part of your system and distribute it throughout the entire code base.
The GroovyDocs are a bit thin on that class, but they say (and I quote), “GroovyTestCase, which recreates internal GroovyShell in each setUp()”. The class has a protected field called shell of type GroovyShell, and a method called withBinding with a couple of overloads. For my purposes, I want the overload that takes two arguments, the first being a Map of binding variables, and the second being a closure to be executed. The method executes the closure with the given binding:
protected def withBinding(Map map, Closure closure)
Therefore, if I extend GroovyShellTestCase, I can rewrite my test as:
class ScriptTests extends GroovyShellTestCase {
void testMath() {
def result = withBinding( [x:3, y:4] ) {
shell.evaluate(new File('math.groovy'))
shell.context.z
}
assertEquals 7, result
}
}
The map supplies
x and y to the binding, which is automatically supplied to the instantiated shell. I can get the result by calling the getContext method on the shell (i.e., access the context property) which returns the binding, and then accessing its z property. I’m taking advantage of the fact that the implementation of the withBinding method includes “return closure.call()“, so the last evaluated expression in the closure is returned automatically.
This isn’t a huge deal, but it’s there and presumably it’s been there for a while and I never knew it. Now I know. Even better, now you know. Maybe you’ll have a use case that needs it. I’ve been trying to make sure that all my scripts in my book have test cases, and this gives me a convenient way to write them.
My only disappointment in discovering this is that when I looked for the tests for the GroovyShellTestCase class, I didn’t find any. In the Groovy distribution under src/test/groovy/util, there’s GroovyScriptEngineTest, GroovyTestCaseTest, and even GroovySwingTestCase, but no dedicated class for testing GroovyShellTestCase. Maybe that’s my opportunity to add one and make a (tiny, but useful) contribution to the language. That would be fund to do just to write a class called GroovyShellTestCaseTest. Heck, that’s only one small step away from the palindromic GroovyShellTestCaseTestShell.groovy.
Friday, November 18, 2011
Way back in the Spring of 2009, I was contacted by an editor at O’Reilly about doing a couple of “targeted video/screencasts”. This person (who is no longer there — I’d give you his name but I haven’t asked his permission yet) had the idea of getting people together in an informal setting and talking about state of the art technologies.
To make a long story short (it’ll get longer later), I wound up flying to Sebastopol, CA in late April, 2009, and we recorded two extended screencasts at the Westerbeke Ranch Conference and Event Center in the heart of Sonoma Valley wine country. We were the only people there, so it was pretty quiet. It also hadn’t yet warmed up, so we were on the chilly side, but that wasn’t a problem either. Oh, and the wine was good, too.
The format of the screencasts was a conversation between me and a single developer, shifting between us and the laptop I was using (a Sony VAIO, with a Camtasia install that almost brought it to its knees).
One of those screencasts, on Enterprise Java, has mercifully been lost to the mists of time. The other now has the title “Up and Running Groovy“, and much to my surprise was released to Safari online today. You can find it at http://my.safaribooksonline.com/video/-/9781449323387.
The code holds up surprisingly well, even after 2 1/2 years. The funniest part, though, is that the second slide mentions the upcoming O’Reilly book “Making Java Groovy“, co-authored by Scott Davis (!) and me.
Um, yeah. The book did start out with Scott, and in late 2009 he signed it over to me. In early 2010, O’Reilly cancelled it, along with many others, citing a lack of demand for Groovy. The people at Manning saw the world differently, and now you can get over half of it through the Manning Early Access Program at http://manning.com/kousen.
So here we are, years later, with the odd situation of an O’Reilly screencast that is effectively marketing a Manning book. I don’t think I could make this stuff up.
If you actually watch it and have any questions, please feel free to send them along. I assure you all questions or comments I’m getting these days are changing how I’m writing my book.


