In my previous blog post, I took a closer look at the numeric type BigInt in JavaScript.
Problem: JSON ¶
I wrote in my previous blog post that JSON does not support BigInt.
If you try to serialize and deserialize an object with BigInt values, the methods JSON.stringify() and JSON.parse() throw errors:
JSON.stringify({a:10n})
// Uncaught TypeError: Do not know how to serialize a BigInt
JSON.parse("{\"a\":10n}")
// Uncaught SyntaxError: Unexpected token n in JSON at position 7
Fortunately, there is a way to work around this problem by converting BigInt to a string and back. I already showed you a JavaScript solution for this in my previous blog post. I will show you this solution again a bit further below, but first, let's look at the problems that the BigInt data type tries to solve.
Problem: Maximum values ¶
In this example, we use a Spring Boot REST controller with a GET handler that creates a POJO with three fields and sends it back to the caller as JSON.
@GetMapping("/fetchData1")
public Payload fetchData1() {
long mersenne8 = (long) (Math.pow(2, 31) - 1);
var mersenne9 = new BigInteger("2").pow(61).subtract(BigInteger.ONE);
return new Payload(1, mersenne8, mersenne9);
}
The Payload POJO consists of three fields. All three data types can store values that may exceed the range that JavaScript can represent safely with the number type.
public class Payload {
private long value1;
private Long value2;
private BigInteger prime;
From JavaScript, we call the service with the Fetch API.
const response1 = await fetch('fetchData1');
const obj1 = await response1.json();
console.log(obj1);
We get back this response:
{value1: 1, value2: 2147483647, prime: 2305843009213694000}
No error is thrown, and at first glance, it looks okay, but prime contains the wrong value. It should be 2305843009213693951 (2^61 - 1). This is a very subtle error because there is no error or warning message. JavaScript cannot accurately represent this integer with the number type. The largest safe integer that JavaScript can represent with number is 2^53 - 1. You can access this value with Number.MAX_SAFE_INTEGER.
Number.MAX_SAFE_INTEGER // 9007199254740991
Number.MAX_SAFE_INTEGER + 1 // 9007199254740992
Number.MAX_SAFE_INTEGER + 2 // 9007199254740992 (wrong!!!)
This is not a bug; it's just how the number type is implemented in JavaScript. See the ECMAScript specification for more information.
Next, we look at an endpoint that returns a number (2^1279 - 1) that is far too large for the number type.
@GetMapping("/fetchData2")
public Payload fetchData2() {
var mersenne15 = new BigInteger("2").pow(1279).subtract(BigInteger.ONE);
return new Payload(2, 9007199254740991L, mersenne15);
}
If we call this endpoint from JavaScript, we get back the following response:
{value1: 2, value2: 9007199254740991, prime: Infinity}
No error is thrown, but we get back this strange Infinity. Infinity is a number in JavaScript, and the JSON parser produces this value when it encounters a JSON number that is too large to represent as a finite number.
There are similar problems when sending objects from a browser to the backend.
@PostMapping("/storeData")
@ResponseStatus(code = HttpStatus.NO_CONTENT)
public void postData(@RequestBody Payload payload) {
System.out.println(payload);
}
From JavaScript, we send an object with a number that is bigger than Number.MAX_SAFE_INTEGER:
const storeObj = {
value1: 3,
value2: 2147483647,
prime: 2305843009213693951
};
fetch('storeData', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(storeObj)
})
On the server, the POST handler prints out this information:
// Payload [value1=3, value2=2147483647, prime=2305843009213694000]
The same error as before. 2305843009213693951 can't be represented precisely in the number type.
JSON.stringify() therefore writes 2305843009213694000 into the JSON and sends that over the wire to the backend.
You can also try and send Infinity:
const storeObj = {
value1: 3,
value2: 2147483647,
prime: Infinity
};
The JSON.stringify() method handles this case by writing null into the JSON:
{"value1":3,"value2":2147483647,"prime":null}
Solution: Sending BigInt as a string in JSON ¶
As a solution for the missing BigInt support in JSON, we convert the BigInts to strings and then convert them back to a numeric type. We do this in JavaScript and in the Spring Boot backend application.
Server: Serialization ¶
To customize the serialization process in Jackson, we have to write subclasses of the com.fasterxml.jackson.databind.JsonSerializer class and override the serialize method.
package ch.rasc.bigint;
import java.math.BigInteger;
import tools.jackson.core.JsonGenerator;
import tools.jackson.databind.SerializationContext;
import tools.jackson.databind.ValueSerializer;
public class BigIntegerSerializer extends ValueSerializer<BigInteger> {
@Override
public void serialize(BigInteger value, JsonGenerator gen, SerializationContext context) {
gen.writeString(value.toString() + "n");
}
}
package ch.rasc.bigint;
import tools.jackson.core.JsonGenerator;
import tools.jackson.databind.SerializationContext;
import tools.jackson.databind.ValueSerializer;
public class LongSerializer extends ValueSerializer<Long> {
@Override
public void serialize(Long value, JsonGenerator gen, SerializationContext context) {
gen.writeString(value.toString() + "n");
}
}
The Long serializer works with the Long object and supports the primitive data type long. Notice that the value parameter passed to the method is never null; we can omit any null checks.
Next, we need to tell Jackson when to use these serializers. In Spring Boot, we can do this in two ways: either globally or locally at the field level.
To install these serializers globally, we create a class that extends the com.fasterxml.jackson.databind.module.SimpleModule class and then call this.addSerializer() in the constructor to register our serializers. Notice that we add the LongSerializer twice: once for handling the object Long and once for the primitive data type long.
We also annotate this class with @Component to denote this class as a Spring-managed bean. Spring Boot automatically picks up this class and configures the ObjectMapper instance used for JSON serialization.
import com.fasterxml.jackson.databind.module.SimpleModule;
@Component
public class CustomModule extends SimpleModule {
public CustomModule() {
this.addSerializer(BigInteger.class, new BigIntegerSerializer());
this.addSerializer(Long.class, new LongSerializer());
this.addSerializer(long.class, new LongSerializer());
}
}
When we call our two endpoints from the previous section, we get back these JSON responses:
/fetchData1
{"value1":"1n","value2":"2147483647n","prime":"2305843009213693951n"}
/fetchData2
{"value1":"2n","value2":"9007199254740991n","prime":"10407932194664399081925240327364085538615262247266...958028878050869736186900714720710555703168729087n"}
Installing serializers globally looks very convenient, but it can lead to problems, especially when you add a serializer for a common type like long. Maybe most long fields will never exceed the range of a JavaScript number in your application, and in these cases, you want to send them as regular numbers over JSON. Instead of configuring the serializers globally, you can configure them directly on a POJO field.
This POJO applies the custom serializers to the fields value2 and prime. You may place @JsonSerialize on the field or the getter method.
public class Payload {
private long value1;
private Long value2;
@JsonSerialize(using = BigIntegerSerializer.class)
private BigInteger prime;
@JsonSerialize(using = LongSerializer.class)
public Long getValue2() {
return this.value2;
}
With this setup, we get the following JSON responses. Notice that value1 is serialized as a regular number because we didn't add a serializer to that field.
/fetchData1
{"value1":1,"value2":"2147483647n","prime":"2305843009213693951n"}
/fetchData2
{"value1":2,"value2":"9007199254740991n","prime":"10407932194664399081925240327364085538615262247266...958028878050869736186900714720710555703168729087n"}
Client: Deserialization ¶
After setting up the server, we now receive strings in the form "2147483647n" that signify a BigInt value.
If we call JSON.parse() on this JSON, we get back a string, but we want an object with a BigInt property.
The JSON.parse() method supports an optional second parameter called reviver. This is a function that is called for each key/value pair. Here, we check whether the value is a string that contains only digits and the letter n at the end. If that is the case, we convert it to a BigInt.
function parseReviver(key, value) {
if (typeof value === 'string' && /^\d+n$/.test(value)) {
return BigInt(value.slice(0, -1));
}
return value;
}
Unfortunately, the json() method of the Fetch response object does not support the reviver argument, so we can't use this pattern.
const response1 = await fetch('fetchData1');
const obj1 = await response1.json();
Instead, we must access the raw string response with text() and then call JSON.parse().
Modern JavaScript runtimes also pass a context.source argument to the reviver for primitive values. That makes it possible to reconstruct large JSON numbers losslessly when the server emits them as JSON numbers. Because this example intentionally uses the "123n" string representation and still needs to work across older runtimes, parsing the response text remains the most portable approach here.
const response1 = await fetch('fetchData1');
const obj1 = JSON.parse(await response1.text(), parseReviver);
console.log(obj1);
// {value1: 1, value2: 2147483647n, prime: 2305843009213693951n}
const response2 = await fetch('fetchData2');
const obj2 = JSON.parse(await response2.text(), parseReviver);
console.log(obj2);
// {value1: 2, value2: 9007199254740991n, prime: 10407932194664399081925240327364085538615262247266…958028878050869736186900714720710555703168729087n}
Client: Serialization ¶
As we have seen in the first section, serializing an object with JSON.stringify() throws an error if it contains properties with BigInt values. Fortunately, this method also supports a second optional parameter called replacer, a function that is called for each key/value pair. In this replacer function, we check the data type and, when it's a BigInt, convert it to a string and append the letter n.
function stringifyReplacer(key, value) {
if (typeof value === 'bigint') {
return value.toString() + 'n';
} else {
return value;
}
}
The following code sends a POST request to our Spring Boot backend and passes the replacer function to JSON.stringify().
const storeObj = {
value1: 3,
value2: 2147483647n,
prime: 2305843009213693951n
};
await fetch('storeData', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(storeObj, stringifyReplacer)
})
This is the JSON the browser sends to the server:
{"value1":3,"value2":"2147483647n","prime":"2305843009213693951n"}
Server: Deserialization ¶
We may customize the deserialization process with the Jackson library by creating subclasses of the com.fasterxml.jackson.databind.JsonDeserializer class.
For this example, I wrote a deserializer for BigInteger, Long, and long.
package ch.rasc.bigint;
import java.math.BigInteger;
import tools.jackson.core.JsonParser;
import tools.jackson.databind.DeserializationContext;
import tools.jackson.databind.ValueDeserializer;
public class BigIntegerDeserializer extends ValueDeserializer<BigInteger> {
@Override
public BigInteger deserialize(JsonParser jp, DeserializationContext ctxt) {
String value = jp.getString();
if (value != null) {
value = value.strip();
if (value.length() > 0) {
return new BigInteger(value.substring(0, value.length() - 1));
}
}
return null;
}
}
package ch.rasc.bigint;
import tools.jackson.core.JsonParser;
import tools.jackson.databind.DeserializationContext;
import tools.jackson.databind.ValueDeserializer;
public class LongDeserializer extends ValueDeserializer<Long> {
@Override
public Long deserialize(JsonParser jp, DeserializationContext ctxt) {
String value = jp.getString();
if (value != null) {
value = value.strip();
if (value.length() > 0) {
return Long.valueOf(value.substring(0, value.length() - 1));
}
}
return null;
}
}
In these two implementations, I check for null and for empty strings. In these two cases, the deserializer returns null; in all other cases, it removes the last letter (n) and tries to convert the string to a BigInteger or a Long, respectively.
You can configure the deserializers, like the serializers, globally or locally on a field-by-field basis.
Globally, you register the deserializers with this.addDeserializer() in a module class.
@Component
public class CustomModule extends SimpleModule {
public CustomModule() {
...
this.addDeserializer(BigInteger.class, new BigIntegerDeserializer());
this.addDeserializer(Long.class, new LongDeserializer());
this.addDeserializer(long.class, new LongDeserializer());
}
}
Alternatively, specify the deserializer with the @JsonDeserialize annotation on the field or the setter method.
public class Payload {
private long value1;
private Long value2;
@JsonDeserialize(using = BigIntegerDeserializer.class)
private BigInteger prime;
@JsonDeserialize(using = LongDeserializer.class)
public void setValue2(Long value2) {
this.value2 = value2;
}
You get these values when the client sends the POST from the previous section. Notice the correct value for prime:
// Payload [value1=3, value2=2147483647, prime=2305843009213693951]
This concludes the tutorial about sending BigInt over JSON.
You can find the complete source code for this example on GitHub:
https://github.com/ralscha/blog2019/tree/master/bigintjson