I want to Javassist as the core JDBC-based Javassist in my free time to write a set of lightweight ORM framework to abandon reflection calls, the process of reading mybatis, tk-mapper, mybatis-plus and spring-boot-starter-jdbc source code. Among them, I found that the LambdaQueryWrapper in mybatis-plus can get the method information (actually CallSite information) of the currently called Lambda expression, so here is a complete record. This article is based on JDK11, other versions of JDK may not be suitable.

Magical Lambda expression serialization

When I was looking at the source code implementation of Lambda expressions, I didn’t look at the comments of LambdaMetafactory, and one of the many comments at the top of this class is as follows.

Search for FLAG_SERIALIZABLE again in the comments of LambdaMetafactory and you can see this comment.

To wit: the function object instances generated after setting the FLAG_SERIALIZABLE flag will implement the Serializable interface, and a method with the name writeReplace will exist with a return value of type SerializedLambda. the methods that call these function objects (the previously mentioned “capture class”) of the caller must exist for a method with the name $deserializeLambda$, as described by the SerializedLambda class.

Finally, looking at the description of SerializedLambda, the annotation has four major paragraphs, which are posted here and extract the core information per small paragraph.

The general idea of each paragraph is as follows.

  • Paragraph 1: SerializedLambda is the serialized form of a Lambda expression; this class stores the runtime information of the Lambda expression
  • Paragraph 2: To ensure that the serialization of a Lambda expression is implemented correctly, one option available to the compiler or language library is to ensure that the writeReplace method returns a SerializedLambda instance
  • Paragraph 3: SerializedLambda provides a readResolve method that functions similarly to calling the static method $deserializeLambda$(SerializedLambda) in the “capture class” and using its own instance as an input, which is understood as a deserialization process
  • Paragraph 4: The identity-sensitive operations of the serialized and deserialized function objects in the form of identifiers (such as System.identityHashCode(), object locking, etc.) are unpredictable.

The ultimate conclusion is that if a functional interface implements the Serializable interface, then its instance automatically generates a writeReplace method that returns a SerializedLambda instance from which the runtime information of the functional interface can be retrieved. This runtime information is the property of SerializedLambda.

Property Meaning
capturingClass “Capture class”, the class in which the current Lambda expression appears
functionalInterfaceClass name, and separated by “/”, the static type of the returned Lambda object
functionalInterfaceMethodName Functional interface method names
functionalInterfaceMethodSignature Functional interface method signature (actually the parameter type and return value type, or the type after erasure if a generic type is used)
implClass Name, and separated by “/”, holding the type of the implementation method of the functional interface method (the implementation class that implements the functional interface method)
implMethodName Implementation method names for functional interface methods
implMethodSignature Method signature of the implementation method of the functional interface method (the real is the parameter type and return value type)
instantiatedMethodType Functional interface types after replacement with instance type variables
capturedArgs Dynamic parameters captured by Lambda
implMethodKind Implement the MethodHandle type of the method

As a practical example, define a functional interface that implements Serializable and call it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

public class App {

    @FunctionalInterface
    public interface CustomerFunction<S, T> extends Serializable {

        T convert(S source);
    }

    public static void main(String[] args) throws Exception {
        CustomerFunction<String, Long> function = Long::parseLong;
        Long result = function.convert("123");
        System.out.println(result);
        Method method = function.getClass().getDeclaredMethod("writeReplace");
        method.setAccessible(true);
        SerializedLambda serializedLambda = (SerializedLambda)method.invoke(function);
        System.out.println(serializedLambda.getCapturingClass());
    }
}

The executed DEBUG message is as follows.

This gives access to the call point runtime information of the functional interface instance when the method is called, and even the type of the generic parameter before it is erased, then many tricks can be derived. For example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

public class ConditionApp {

    @FunctionalInterface
    public interface CustomerFunction<S, T> extends Serializable {

        T convert(S source);
    }

    @Data
    public static class User {

        private String name;
        private String site;
    }

    public static void main(String[] args) throws Exception {
        Condition c1 = addCondition(User::getName, "=", "throwable");
        System.out.println("c1 = " + c1);
        Condition c2 = addCondition(User::getSite, "IN", "('throwx.cn','vlts.cn')");
        System.out.println("c1 = " + c2);
    }

    private static <S> Condition addCondition(CustomerFunction<S, String> function,
                                              String operation,
                                              Object value) throws Exception {
        Condition condition = new Condition();
        Method method = function.getClass().getDeclaredMethod("writeReplace");
        method.setAccessible(true);
        SerializedLambda serializedLambda = (SerializedLambda) method.invoke(function);
        String implMethodName = serializedLambda.getImplMethodName();
        int idx;
        if ((idx = implMethodName.lastIndexOf("get")) >= 0) {
            condition.setField(Character.toLowerCase(implMethodName.charAt(idx + 3)) + implMethodName.substring(idx + 4));
        }
        condition.setEntityKlass(Class.forName(serializedLambda.getImplClass().replace("/", ".")));
        condition.setOperation(operation);
        condition.setValue(value);
        return condition;
    }

    @Data
    private static class Condition {

        private Class<?> entityKlass;
        private String field;
        private String operation;
        private Object value;
    }
}

// 执行结果
c1 = ConditionApp.Condition(entityKlass=class club.throwable.lambda.ConditionApp$User, field=name, operation==, value=throwable)
c1 = ConditionApp.Condition(entityKlass=class club.throwable.lambda.ConditionApp$User, field=site, operation=IN, value=('throwx.cn','vlts.cn'))

Many people will worry about the performance of reflection calls, in fact, in high versions of the JDK, reflection performance has been greatly optimized, very close to the performance of direct calls, not to mention that some scenarios are a small number of reflection calls scenario, you can use with confidence.

Having spent a lot of time showing the functionality and use of SerializedLambda, let’s look at serialization and deserialization of Lambda expressions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public class SerializedLambdaApp {

    @FunctionalInterface
    public interface CustomRunnable extends Serializable {

        void run();
    }

    public static void main(String[] args) throws Exception {
        invoke(() -> {
        });
    }

    private static void invoke(CustomRunnable customRunnable) throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(baos);
        oos.writeObject(customRunnable);
        oos.close();
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(baos.toByteArray()));
        Object target = ois.readObject();
        System.out.println(target);
    }
}

The results are shown in the following figure.

Lambda expressions serialization principle

Regarding the principle of Lambda expression serialization, you can directly refer to the source code of ObjectStreamClass, ObjectOutputStream and ObjectInputStream, and here is the direct conclusion.

  • prerequisite: the object to be serialized needs to implement the Serializable interface
  • to be serialized object if there is writeReplace method, then directly based on the incoming instance reflection call this method to get the return value type as the target type of serialization, for Lambda expressions is SerializedLambda type
  • The process of deserialization happens to be the process of reversal, and the method called is readResolve, which happens to have a private method of the same name as mentioned earlier for SerializedLambda
  • The implementation type of the Lambda expression is the VM-generated template class, and as observed from the result, the instance before serialization and the instance obtained after deserialization belong to different template classes, for the example in the previous subsection the template class before serialization in the result of a certain run is club.throwable.lambda.SerializedLambdaApp$$ Lambda$$14/0x0000000800065840, and after deserialization the template class is club.throwable.lambda.SerializedLambdaApp$$Lambda$$26/0x00000008000a4040

ObjectStreamClass is the class descriptor for serialization and deserialization implementations. Information about the class descriptor for object serialization and deserialization can be found in the member properties of this class, such as the writeReplace and readResolve methods mentioned here

The graphical process is as follows.

Get SerializedLambda’s way

From the previous analysis, it is known that there are two ways to obtain a SerializedLambda instance of a Lambda expression.

  • Way 1: Call the writeReplace method based on the reflection of the Lambda expression instance and the template class of the Lambda expression, and the return value obtained is the SerializedLambda instance
  • Way 2: Get SerializedLambda instance based on serialization and deserialization

Based on these two ways you can write separate examples, for example the reflection way as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

// 反射方式
public class ReflectionSolution {

    @FunctionalInterface
    public interface CustomerFunction<S, T> extends Serializable {

        T convert(S source);
    }

    public static void main(String[] args) throws Exception {
        CustomerFunction<String, Long> function = Long::parseLong;
        SerializedLambda serializedLambda = getSerializedLambda(function);
        System.out.println(serializedLambda.getCapturingClass());
    }

    public static SerializedLambda getSerializedLambda(Serializable serializable) throws Exception {
        Method writeReplaceMethod = serializable.getClass().getDeclaredMethod("writeReplace");
        writeReplaceMethod.setAccessible(true);
        return (SerializedLambda) writeReplaceMethod.invoke(serializable);
    }
}

The serialized and deserialized approach is slightly more complicated because the ObjectInputStream.readObject() method will end up calling back the SerializedLambda.readResolve() method, resulting in the return of a new template class hosting an instance of the Lambda expression, so here we need to find a way to interrupt this The solution is to construct a shadow type similar to SerializedLambda but without the readResolve() method.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

package cn.vlts;
import java.io.Serializable;

/**
 * 这里注意一定要和java.lang.invoke.SerializedLambda同名,可以不同包名,这是为了"欺骗"ObjectStreamClass中有个神奇的类名称判断classNamesEqual()方法
 */
@SuppressWarnings("ALL")
public class SerializedLambda implements Serializable {
    private static final long serialVersionUID = 8025925345765570181L;
    private  Class<?> capturingClass;
    private  String functionalInterfaceClass;
    private  String functionalInterfaceMethodName;
    private  String functionalInterfaceMethodSignature;
    private  String implClass;
    private  String implMethodName;
    private  String implMethodSignature;
    private  int implMethodKind;
    private  String instantiatedMethodType;
    private  Object[] capturedArgs;

    public String getCapturingClass() {
        return capturingClass.getName().replace('.', '/');
    }
    public String getFunctionalInterfaceClass() {
        return functionalInterfaceClass;
    }
    public String getFunctionalInterfaceMethodName() {
        return functionalInterfaceMethodName;
    }
    public String getFunctionalInterfaceMethodSignature() {
        return functionalInterfaceMethodSignature;
    }
    public String getImplClass() {
        return implClass;
    }
    public String getImplMethodName() {
        return implMethodName;
    }
    public String getImplMethodSignature() {
        return implMethodSignature;
    }
    public int getImplMethodKind() {
        return implMethodKind;
    }
    public final String getInstantiatedMethodType() {
        return instantiatedMethodType;
    }
    public int getCapturedArgCount() {
        return capturedArgs.length;
    }
    public Object getCapturedArg(int i) {
        return capturedArgs[i];
    }
}


public class SerializationSolution {

    @FunctionalInterface
    public interface CustomerFunction<S, T> extends Serializable {

        T convert(S source);
    }

    public static void main(String[] args) throws Exception {
        CustomerFunction<String, Long> function = Long::parseLong;
        cn.vlts.SerializedLambda serializedLambda = getSerializedLambda(function);
        System.out.println(serializedLambda.getCapturingClass());
    }

    private static cn.vlts.SerializedLambda getSerializedLambda(Serializable serializable) throws Exception {
        try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
             ObjectOutputStream oos = new ObjectOutputStream(baos)) {
            oos.writeObject(serializable);
            oos.flush();
            try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(baos.toByteArray())) {
                @Override
                protected Class<?> resolveClass(ObjectStreamClass desc) throws IOException, ClassNotFoundException {
                    Class<?> klass = super.resolveClass(desc);
                    return klass == java.lang.invoke.SerializedLambda.class ? cn.vlts.SerializedLambda.class : klass;
                }
            }) {
                return (cn.vlts.SerializedLambda) ois.readObject();
            }
        }
    }
}

The forgotten $deserializeLambda$ method

As mentioned earlier, the deserialization of a Lambda expression instance calls the java.lang.invoke.SerializedLambda.readResolve() method, which, magically, has the following source code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

private Object readResolve() throws ReflectiveOperationException {
    try {
        Method deserialize = AccessController.doPrivileged(new PrivilegedExceptionAction<>() {
            @Override
            public Method run() throws Exception {
                Method m = capturingClass.getDeclaredMethod("$deserializeLambda$", SerializedLambda.class);
                m.setAccessible(true);
                return m;
            }
        });

        return deserialize.invoke(null, this);
    }
    catch (PrivilegedActionException e) {
        Exception cause = e.getException();
        if (cause instanceof ReflectiveOperationException)
            throw (ReflectiveOperationException) cause;
        else if (cause instanceof RuntimeException)
            throw (RuntimeException) cause;
        else
            throw new RuntimeException("Exception in SerializedLambda.readResolve", e);
    }
}

It looks like there is a static method in the “capture class” that says

1
2
3
4
5
6

class CapturingClass {

    private static Object $deserializeLambda$(SerializedLambda serializedLambda){
        return [serializedLambda] => Lambda表达式实例;
    }  
}

You can try to retrieve the list of methods in the “capture class”.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

public class CapturingClassApp {

    @FunctionalInterface
    public interface CustomRunnable extends Serializable {

        void run();
    }

    public static void main(String[] args) throws Exception {
        invoke(() -> {
        });
    }

    private static void invoke(CustomRunnable customRunnable) throws Exception {
        Method writeReplaceMethod = customRunnable.getClass().getDeclaredMethod("writeReplace");
        writeReplaceMethod.setAccessible(true);
        java.lang.invoke.SerializedLambda serializedLambda = (java.lang.invoke.SerializedLambda)
                writeReplaceMethod.invoke(customRunnable);
        Class<?> capturingClass = Class.forName(serializedLambda.getCapturingClass().replace("/", "."));
        ReflectionUtils.doWithMethods(capturingClass, method -> {
                    System.out.printf("方法名:%s,修饰符:%s,方法参数列表:%s,方法返回值类型:%s\n", method.getName(),
                            Modifier.toString(method.getModifiers()),
                            Arrays.toString(method.getParameterTypes()),
                            method.getReturnType().getName());
                },
                method -> Objects.equals(method.getName(), "$deserializeLambda$"));
    }
}

// 执行结果
方法名:$deserializeLambda$,修饰符:private static,方法参数列表:[class java.lang.invoke.SerializedLambda],方法返回值类型:java.lang.Object

SerializedLambda annotation description is consistent with the previously mentioned java.lang.invoke. I guess $deserializeLambda$ is a method generated by the VM and can only be called by reflection, so it is a deeper hidden trick.

Summary

The Lambda expression feature in the JDK has been released for many years, and I can’t imagine that it took so many years to figure out its serialization and deserialization methods today. Although this is not a complex problem, it is one of the more interesting points of knowledge that I have seen in recent times.