0x01 Preface
This article is detailed. Copy the demo and follow it in a debugger step by step for easier understanding.
0x02 Flow Analysis
Serialization is the process of writing an object to an I/O stream. It usually begins by creating anObjectOutputStreamoutput stream, then callObjectOutputStreamobject'swriteObjectoutputs the serializable object in the specified format described above.
Demo:
package com.panda.alipay;
import java.io.*;
public class Main {
public static class Demo implements Serializable {
private String string;
transient String name = "hello";
public Demo(String s) {
this.string = s;
}
public static void main(String[] args) throws IOException {
Demo demo = new Demo("panda");
ObjectOutputStream outputStream = new ObjectOutputStream(new FileOutputStream("panda.out"));
outputStream.writeObject(new Demo("panda"));
outputStream.close();
}
}
}
The two most important lines are:
ObjectOutputStream outputStream = new ObjectOutputStream(new FileOutputStream("panda.out"));
outputStream.writeObject(new Demo("panda"));
These two lines contain the entire serialization flow.
First examineObjectOutputStream，ObjectOutputStreamimplementsObjectOutputinterface'sOutputStreamsubclass, defined as:
public class ObjectOutputStream
extends InputStream implements ObjectInput, ObjectStreamConstants{
...
}
When we instantiateObjectOutputStreamwith the argument, first callObjectOutputStreamconstructor.
ObjectOutputStreamhas two constructors, onepublicsingle-argument constructor and oneprotectedno-argument constructor; the code above receivesnew FileOutputStream("panda.out")as the argument, so it callsObjectOutputStreamofpublicsingle-argument constructor:
/**
* Creates an ObjectOutputStream that writes to the specified OutputStream.
* This constructor writes the serialization stream header to the underlying stream;
* callers may wish to flush the stream immediately so that constructors of receiving ObjectInputStreams do not block while reading the header.
* If a security manager is installed, this constructor checks for the "enableSubclassImplementation" SerializablePermission when it is called directly or indirectly by the constructor of a subclass that overrides ObjectOutputStream.putFields or ObjectOutputStream.writeUnshared.
*/
public ObjectOutputStream(OutputStream out) throws IOException {
verifySubclass();
bout = new BlockDataOutputStream(out);
handles = new HandleTable(10, (float) 3.00);
subs = new ReplaceTable(10, (float) 3.00);
enableOverride = false;
writeStreamHeader();
bout.setBlockDataMode(true);
if (extendedDebugInfo) {
debugInfoStack = new DebugTraceInfoStack();
} else {
debugInfoStack = null;
}
}
At constructor start, first callverifySubclassprocesses cached information and requires the class or subclass to validate that the instance can be constructed without violating security constraints.
Then initializeboutand others, instantiate aBlockDataOutputStream；
Question:boutand others?BlockDataOutputStreamWhat is it, and why initialize it here?boutmember field?
1、boutand others?
boutis a field of the main class. BesidesboutOther fields includehandles: hash table mapping objects to references;subs: a hash table mapping objects to replacement objects;enableOverride: Boolean constant selecting which method serializes Java objectswriteObjectOverridemethod orwriteObjectmethod.
/** filter stream for handling block data conversion */
private final BlockDataOutputStream bout;
/** obj -> wire handle map */
private final HandleTable handles;
/** obj -> replacement obj map */
private final ReplaceTable subs;
/** if true, invoke writeObjectOverride() instead of writeObject() */
private final boolean enableOverride;
We can treatbout can be viewed as a container: a filtering stream that transforms data blocks.
2、BlockDataOutputStreamWhat is it?
BlockDataOutputStreamisObjectOutputStreamimportant internal class responsible for writing buffered data to the byte stream. Part of it is:
/*
Buffered output stream with two modes: in default mode it writes data in the same format as DataOutputStream; in "block data" mode it writes data bracketed by block data markers (see the Object Serialization Specification for details).
*/
private static class BlockDataOutputStream extends OutputStream implements DataOutput
{
/** maximum data block length */
private static final int MAX_BLOCK_SIZE = 1024;
/** maximum data block header length */
private static final int MAX_HEADER_SIZE = 5;
/** (tunable) length of char buffer (for writing strings) */
private static final int CHAR_BUF_SIZE = 256;
/** buffer for writing general/block data */
private final byte[] buf = new byte[MAX_BLOCK_SIZE];
/** buffer for writing block data headers */
private final byte[] hbuf = new byte[MAX_HEADER_SIZE];
/** char buffer for fast string writes */
private final char[] cbuf = new char[CHAR_BUF_SIZE];
/** block data mode */
private boolean blkmode = false;
/** current offset into buf */
private int pos = 0;
/** underlying output stream */
private final OutputStream out;
/** loopback stream (for data writes that span data blocks) */
private final DataOutputStream dout;
/**
* Creates new BlockDataOutputStream on top of given underlying stream.
* Block data mode is turned off by default.
*/
BlockDataOutputStream(OutputStream out) {
this.out = out;
dout = new DataOutputStream(this);
}
......
}
This class resembles the main class (ObjectOutputStream) definition, except for the implemented interface.
It can be understood asBlockDataOutputStreamclass is a wrappedDataOutputStreamclass, providing buffers and member fields.
3. Why initializeboutmember field?
writeObject0method primarily usesboutobject methodssetBlockDataModeDisableData Blockmode;
Data Blockmode:
JDK 1.2 needed a byte-stream format incompatible with JDK 1.1 while preserving forward compatibility. A compatibility marker like
PROTOCOL_VERSIONformat,ObjectOutputStreaminuseProtocolVersionaccepts an argument selecting the serializable byte-stream protocol version.The byte-stream protocol versions are:
ObjectStreamConstants.PROTOCOL_VERSION_1: original serialized byte-stream format;ObjectStreamConstants.PROTOCOL_VERSION_2: new external stream format, writing primitive data in blocks [Data-Block] mode into the byte stream, beginning with markerTC_ENDBLOCKDATAEndData-block boundaries are standardized. Primitive data written in block mode normallycannot exceed 1024bytes long. This standardizes serialized format and improves forward and backward compatibility.
JDK1.2Defaults toPROTOCOL_VERSION_2JDK1.1Defaults toPROTOCOL_VERSION_1JDK 1.1.7and later can read both versions, whileJDK 1.1.7Earlier versions can read onlyPROTOCOL_VERSION_1version;
See the original Object Serialization Stream Protocol:https://docs.oracle.com/javase/8/docs/platform/serialization/spec/protocol.html
Or see my translated summary Object Serialization Stream Protocol: /talksafe/892.html
Returning to the main flow, after fields initialize, callwriteStreamHeader()method. Following it shows that it is used forObjectOutputStreamduring instance initialization intoboutvariable receives the magic header and version:
WhenObjectOutputStreamofpublicOnly after the constructor completes does it callwriteObject()begins writing object data. Its core code is:
public final void writeObject(Object obj) throws IOException {
if (enableOverride) {
writeObjectOverride(obj);
return;
}
try {
writeObject0(obj, false);
} catch (IOException ex) {
if (depth == 0) {
writeFatalException(ex);
}
throw ex;
}
}
Generally,enableOverridedefault value isfalse
(because inObjectOutputStreamofpublicconstructor has initializedenableOverride = false;）
Then enters the key methodwriteObject0continues serialization. The method is lengthy:
/**
* Underlying writeObject/writeUnshared implementation.
*/
private void writeObject0(Object obj, boolean unshared)
throws IOException
{
boolean oldMode = bout.setBlockDataMode(false);
depth++;
try {
// handle previously written and non-replaceable objects
int h;
if ((obj = subs.lookup(obj)) == null) {
writeNull();
return;
} else if (!unshared && (h = handles.lookup(obj)) != -1) {
writeHandle(h);
return;
} else if (obj instanceof Class) {
writeClass((Class) obj, unshared);
return;
} else if (obj instanceof ObjectStreamClass) {
writeClassDesc((ObjectStreamClass) obj, unshared);
return;
}
// check for replacement object
Object orig = obj;
Class<?> cl = obj.getClass();
ObjectStreamClass desc;
for (;;) {
// REMIND: skip this check for strings/arrays?
Class<?> repCl;
desc = ObjectStreamClass.lookup(cl, true);
if (!desc.hasWriteReplaceMethod() ||
(obj = desc.invokeWriteReplace(obj)) == null ||
(repCl = obj.getClass()) == cl)
{
break;
}
cl = repCl;
}
if (enableReplace) {
Object rep = replaceObject(obj);
if (rep != obj && rep != null) {
cl = rep.getClass();
desc = ObjectStreamClass.lookup(cl, true);
}
obj = rep;
}
// if object replaced, run through original checks a second time
if (obj != orig) {
subs.assign(orig, obj);
if (obj == null) {
writeNull();
return;
} else if (!unshared && (h = handles.lookup(obj)) != -1) {
writeHandle(h);
return;
} else if (obj instanceof Class) {
writeClass((Class) obj, unshared);
return;
} else if (obj instanceof ObjectStreamClass) {
writeClassDesc((ObjectStreamClass) obj, unshared);
return;
}
}
// remaining cases
if (obj instanceof String) {
writeString((String) obj, unshared);
} else if (cl.isArray()) {
writeArray(obj, desc, unshared);
} else if (obj instanceof Enum) {
writeEnum((Enum<?>) obj, desc, unshared);
} else if (obj instanceof Serializable) {
writeOrdinaryObject(obj, desc, unshared);
} else {
if (extendedDebugInfo) {
throw new NotSerializableException(
cl.getName() + "\n" + debugInfoStack.toString());
} else {
throw new NotSerializableException(cl.getName());
}
}
} finally {
depth--;
bout.setBlockDataMode(oldMode);
}
}
and analyze step by step.
inwriteObject0()At the beginning of the method:
boolean oldMode = bout.setBlockDataMode(false);
First disable output-streamData Blockmode and setOriginal modeassign to variableoldMode，
Then enter this decision block:
As its comment says, the code block above handlesalready processedandnon-replaceableobjects cannot be serialized. Most code never enters this block.
Specifically, code first enterssubs.lookup(obj)to decide:
According to the method description—Finds and returns a replacement for the object, or the original object if no replacement exists.
This handles previously written and non-replaceable objects. More simply, it checks whether the incoming object exists in the replacement table (ReplaceTable) table, callwriteNullmethod.
Continue checking whether the mode isunsharedmethod, immediately followed by handles.lookup(obj). Following it reveals:
Thislookupfinds and returns thehandler. If no mapping exists, return -1—checking whether it is in the reference table (HandleTable) table; if present, callwriteHandlemethod and return. If absent, return -1 and serialize further.
Continue following:
Check whether the incoming object is the special typeClassandObjectStreamClass. If so, callwriteClassorwriteClassDescmethod and returns;
If none of the conditions match, check whether object replacement is enabled.
As shown, inspect member fieldenableReplacevalue determines whether replacement is enabledReplace) feature;
But in factenableReplacevalue is usuallyfalse
This code path is not entered.
Then enter the second check:
If replaced, check the original object again, similarly to the initial code. Insert the replacement intosubs(replacement table), then perform similar checks.
After the steps above, process the remaining object types:
If the object is a String, callwriteStringwrites data to the byte stream;
If the object is an Array, callwriteArraywrites data to the byte stream;
If the object is an Enum, callwriteEnumwrites data to the byte stream;
If the object implementsSerializableinterface, callwriteOrdinaryObjectwrites data to the byte stream;
If none match, throwNotSerializableExceptionexception information;
ForwriteString、writeArray、writeEnumwill not be discussed in detail; usewriteStringas a brief example.
private void writeString(String str, boolean unshared) throws IOException {
handles.assign(unshared ? null : str);
long utflen = bout.getUTFLength(str);
if (utflen <= 0xFFFF) {
bout.writeByte(TC_STRING);
bout.writeUTF(str, utflen);
} else {
bout.writeByte(TC_LONGSTRING);
bout.writeLongUTF(str, utflen);
}
}
Before writing a String, the code checks whether the write mode isunshared. If it is notunsharedapproach also requireshandlesobject map, insert the current String, then callgetUTFLengthgets String length and0xFFFFthreshold. Above it, the String is long, so first writeTC_LONGSTRINGmarker (LONGSTRING), then string length and contents. At or below the threshold, it is a normal String and first writesTC_STRINGmarker (STRING), then writes length and contents;
Now focus onwriteOrdinaryObjectmethod.
Before writing obj, first callcheckSerialize()Check whether the current object is serializable; otherwise stop and thrownewInvalidClassException()error:
If serializable, begin writingTC_OBJECTmarker (start), then callwriteClassDescwrites the current object's class descriptor. Follow it:
writeClassDescselects how the class descriptor is written. If it is a null reference, callwriteNullmethod; if not usingunsharedmethod and can inhandlesobject pool contains the incoming object, callwriteHandle. If the class is a dynamic proxy, callwriteProxyDescmethod. If none of the three conditions match, callwriteNonProxyDescmethod.
writeProxyDescandwriteStringis similar and is not reached in this demo, so it is omitted.
ExaminewriteNonProxyDesc：
First writeTC_CLASSDESCmarker (start of new class descriptor), then determine whether the mode isunsharedmode, assigndescclass metadata intohandlesobject map, then calls a write method based on stream protocol version. If the protocol isPROTOCOL_VERSION_1, directly calldescmember'swriteNonProxymethod and assign the current referencethisas an argument towriteNonProxymethod. If not usingPROTOCOL_VERSION_1protocol, call the current class'swriteClassDescriptormethod.
callswriteNonProxymethod. Follow it:
First callwriteUTFmethod writesclass nameto the byte stream. The class name is fully qualified, including package (out.writeUTF(name);）
Then callwriteLongmethod writesserialVersionUIDvalue to the byte stream ( out.writeLong(getSerialVersionUID());）
Then write the number of fields in the current class to the stream (out.writeShort(fields.length);）
Finally, each field's information is written. It has three parts:TypeCode、fieldName、fieldType
Debugging ends here:
Next enableData Blockmode, then callannotateClassmethod,annotateClasshas no concrete implementation:
is provided for subclasses and normally does nothing. A similar method isObjectInputStreaminresolveClassmethod.
When callingannotateClassAfter the method completes, disableData Blockmode, then writeTC_ENDBLOCKDATAmarker (end of non-proxy class descriptor)
At this point,writeNonProxyandwriteClassDescriptorFlow ends, which likewise causeswriteClassDescFlow ends, then return towriteOrdinaryObjectmethod.
Continue withwriteOrdinaryObjectthe code below
If the selected mode isunsharedmode, assigndescclass metadata intohandlesobject map, then checks the current Java object's serialization semantics. If it is not aDynamic proxy classand implementsExternalizable, callwriteExternalDatawrites object information. If the object implementsSerializableinterface, callwriteSerialDatawrites object information.
writeExternalDataCore code:
private void writeExternalData(Externalizable obj) throws IOException {
PutFieldImpl oldPut = curPut;
curPut = null;
if (extendedDebugInfo) {
debugInfoStack.push("writeExternal data");
}
SerialCallbackContext oldContext = curContext;
try {
curContext = null;
if (protocol == PROTOCOL_VERSION_1) {
obj.writeExternal(this);
} else {
bout.setBlockDataMode(true);
obj.writeExternal(this);
bout.setBlockDataMode(false);
bout.writeByte(TC_ENDBLOCKDATA);
}
} finally {
curContext = oldContext;
if (extendedDebugInfo) {
debugInfoStack.pop();
}
}
curPut = oldPut;
}
First checks the byte-stream protocol. If it usesPROTOCOL_VERSION_1protocol, directly call the serializable object'swriteExternalmethod. If not usingPROTOCOL_VERSION_1protocol, first enableData Blockmode, then callwriteExternalmethod, then disablesData Blockmode and append at the stream endTC_ENDBLOCKDATAmarker.
This method switches context: before checking the protocol, it first setscurPutandcurContext is empty. After checking and writing data, set eachcurContext curPuttooldContextandoldPut, restoring the previous environment.
Question: why switch context here?
Now examinewriteSerialData. This writes field values and references into obj, recursively from the highest superclass downward. Detailed flow:
Before serializing the object, obtain from its class descriptorClassDataSlotinformation; after obtaining the inheritance hierarchy, iterate it.
First check whether the serializable object overrideswriteObjectmethod. If overridden, first enableData Blockmode, then callwriteObjectmethod, then disablesData Blockmode and finally appendTC_ENDBLOCKDATAmarker (end of data block). If the method is not overridden, calldefaultWriteFieldswrites all field information. FollowdefaultWriteFieldsmethod:
defaultWriteFieldsreads field data from obj (desc) and writes field data to the byte stream. Flow:
First usecheckDefaultSerialize()Check whether the current object is serializable.
If the object is not serializable, thrownewInvalidClassExceptionexception.
After checking, retrieve every primitive field value.
entersgetPrimFieldValuesinside the methodgetPrimFieldValuesmethod:
Primitive field types are:
After obtaining primitive field values, the system writes them to the byte stream.
At the end of writing, the system callswriteObject0method:
This method writes object-type field values, completing serialization.
Approximate call stack:
Finally, review the complete serialization flow in the diagram:
0x03 Conclusion
Serialization is simple in principle: severalwrite*method:writeFataException、writeNull、writeHandle、writeClass、writeProxyDesc、writeNonProxyDesc、writeString、writeArray、writeEnum, plus two specialwrite*method:writeExternalData、writeOrginaryObject。
Serialization is complex, with many branches and features, including fields markedtransientfields have 'do not serialize' semantics and are ignored during serialization bystaticfields belong to the class rather than the object and are likewise ignored during serialization.
Overall, understanding any complete serialization path (reaching the finalwrite*) helps explain serialization mechanics.
0x04 References
https://docs.oracle.com/javase/7/docs/platform/serialization/spec/serialTOC.html
https://blog.csdn.net/silentbalanceyh/article/details/8294269