Sunday, December 1, 2013

Library to make null-safe collections in thrift generated java object.


While I was experimenting evernote java API, I found thrift generated code for container types(list,set,map) are not really java friendly, especially for for-loop.
In thrift, when value is set to null, the transport will not send the field. But in java, when collection is null, the enhanced-for-loop throws NullPointerException.
Note note = noteStore.getNote(...);
// This throws NPE when the note doesn't have any resources
for (Resource resource : note.getResources()) {
...
}

I can put if-statement to check the collection whenever I need to access it, but that's just repetitive...

So, I wrote a util class that wraps thrift generated class to make it null-safe.

* w() is a static method in ThriftWrapper class which I wrote.
Note wrapped = w(new Note());  // wrapped
for (Resource resource : wrapped.getResources()) {
  // no more NPE. nested thrift attributes are also null-safe
}

Mechanism:

The wrapping method introspect the given thrift object and set empty mutable collections(ArrayList, HashSet, HashMap) when it finds null in collection fields. Also, it traverse child thrift classes and set empty collections as well.(foo.getBar().getBaz().getList()).
Then it returns cglib generated proxy. Since in thrift, null has a meaning when it serializes object, the proxy keeps track of collection fields which were initially null, and when "write()" method is called, it compares the current value and if it's still empty, then it will omit the field to transport.

wrapping behavior:
Note note = new Note();
note.setAttributes(new NoteAttributes());
assertThat(note.getTagGuids(), is(nullValue()));
assertThat(note.getAttributes().getClassifications(), is(nullValue()));

Note wrapped = w(note);
assertThat(wrapped.getTagGuids(), is(emptyCollectionOf(String.class)));
assertThat(wrapped.getAttributes().getClassifications(), is(notNullValue()));
assertThat(wrapped.getAttributes().getClassifications().size(), is(0));
transport behavior:
Note note = new Note();
Note wrapped = w(note);  // still a note instance  (actually a cglib proxy)
wrapped.getTagGuids().size();  // 0 (empty list)
wrapped.getResources().add(...);  // add something to collection

noteStore.createNote(wrapped);  // internally call wrapped.write(...)
// added resources are included in message, but tag-guids are not

Please reference more detailed behavior in test classes.


Code:

I pushed to my github repo.


Further:

Since I'm new to thrift, it might not be a good approach or there might be a better solution already.
If there is more request for this approach, I'll further enhance the project such:

  • inline cglib to avoid dependency conflict
  • make the class more spring-framework friendly bean instead of static methods
  • release and push to public maven repo
  • etc.


Wednesday, October 30, 2013

Spring Boot Actuator: How to change the endpoint base url


By default, actuator management endpoints are mapped to the top level url: "/info", "/metrics", "/beans", etc.
That may be good. But probably it would be better to have some base path. For example, "/admin/metrics" or "/manage/info".
Spring Boot provides quick and easy solution.

Configuration

In your configuration resource file (application.properties or application.yml): 
management.contextPath: /admin
Now, all the management endpoints are mapped under "/admin" base path.
"/admin/metrics", "/admin/info", "/admin/health"...

Implementation Classes

In spring boot actuator source code:
  • org.springframework.boot.actuate.properties.ManagementServerProperties
    • property values representation class
  • org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration
    • @Configuration which uses above ManagementServerProperties


Friday, October 18, 2013

How spring-boot enables actuator by adding jar dependency?


I was reading spring-boot source code, and wondered how it is auto-detecting actuator features just by adding jar dependency.


spring-boot-actuator


"spring-boot-actuator" adds management-endpoints(http) to the spring-boot application just by adding the jar file.

endpoints are metrics, health-check, spring-security auth audit, etc.

please see here for feature details.


To add actuator to your spring-boot application, add this dependencies:


    org.springframework.boot
    spring-boot-starter-actuator


    org.springframework.boot
    spring-boot-starter-web


Mechanism of auto-detecting spring-boot-actuator


So, how spring-boot application detects spring-boot-actuator?

Sample spring-boot application:

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
}

"org.springframework.boot.SpringApplication" is the starting point for spring-boot application.

Auto-detection Steps

  1. During "SpringApplication#run" method, it creates an application context from the given source classes("@Configuration" annotated classes). In this case "MyApplication" class.
  2. When "MyApplication" class is being processed as a spring's Java-configuration, "@EnableAutoConfiguration" annotation gets interpreted. "@EnableAutoConfiguration" is a spring-boot(autoconfigure) annotation which is handled by "org.springframework.boot.autoconfigure.EnableAutoConfigurationImportSelector". (how spring handles @Enable* annotation is well documented in this blog post.)
  3. In "EnableAutoConfigurationImportSelector", it uses "org.springframework.core.io.support.SpringFactoriesLoader#loadFactoryNames" from spring-core to load configurations whose key is  "org.springframework.boot.autoconfigure.EnableAutoConfiguration".
    This method reads "META-INF/spring.factories" from jar files.(multiple jar files can have "spring.factories" and when they have same key, comma delimited values will be merged.)
    spring-boot-actuator contains "META-INF/spring.factories" file. The value of the file is a comma delimited list of "@Configuration" classes under "org.springframework.boot.actuate.autoconfigure" package, which are actuator bean definition classes.
    org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
    org.springframework.boot.actuate.autoconfigure.AuditAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.EndpointAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.ErrorMvcAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.ManagementServerPropertiesAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.MetricFilterAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.MetricRepositoryAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.SecurityAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.TraceRepositoryAutoConfiguration,\
    org.springframework.boot.actuate.autoconfigure.TraceWebFilterAutoConfiguration
  4. These configuration classes will be imported into the main application-context, so that, spring-boot-actuator beans become available in the application.

Same mechanism is used for "spring-boot-autoconfigure"
Its jar file contains "spring.factories" file.


Writing a Custom Spring-Boot Module


In sum, you can write a spring-boot module which will be auto-detected by spring-boot application based on the presence of the jar file:
  • include "META-INF/spring.factories" file
    • key: "org.springframework.boot.autoconfigure.EnableAutoConfiguration"
    • value: comma delimited path to @Configuration file(s) for your module








Thursday, October 10, 2013

released datasource-proxy v1.2


I released datasource-proxy version 1.2

main feature:
  • query and parameter replacement
    • new QueryTransformer and ParameterTransformer APIs to transform query and parameter before executing queries
  • many refactoring

They are available in maven central.

dependency:

 project homepage: https://github.com/ttddyy/datasource-proxy