Working with MongoDB TTL Indexes from Java

If you only need automatic expiration based on time, TTL indexes are the simplest MongoDB tool for the job. They work on regular collections, support normal queries, and are usually a better fit than capped collections for age-based retention.

The examples in this post use MongoCollection.createIndex() together with Indexes and IndexOptions from the MongoDB Java Sync Driver.

Create a TTL Index ¶

You can create a TTL index on a single date field or on a date array field. The TTL value is the amount of time MongoDB should keep a document after the indexed date.

      // TTL Index
      collection.createIndex(Indexes.ascending("date"),
          new IndexOptions().expireAfter(1L, TimeUnit.MINUTES));

Ttl1.java

The sample then inserts five log entries with the current timestamp and waits for them to expire.

      MongoCollection<Document> collection = db.getCollection("log");

      // TTL Index
      collection.createIndex(Indexes.ascending("date"),
          new IndexOptions().expireAfter(1L, TimeUnit.MINUTES));

      for (int j = 0; j < 5; j++) {
        Document logMessage = new Document();
        logMessage.append("date", Date.from(Instant.now()));
        logMessage.append("severity", "INFO");
        logMessage.append("message", String.valueOf(j));
        collection.insertOne(logMessage);
      }

      System.out.println(collection.countDocuments()); // 5

      TimeUnit.SECONDS.sleep(120);

      System.out.println(collection.countDocuments()); // 0

Ttl1.java

TTL indexes support queries like regular indexes. If the indexed field is an array, MongoDB uses the earliest date in the array to compute expiration. If the field is missing or does not contain a date, the document never expires.

MongoDB checks TTL indexes every 60 seconds. TTL deletions run only on the primary in a replica set, and starting with MongoDB 6.1 the server may batch those deletes. Because of that, expiration is not immediate. On an idle system, the extra delay can be only a few seconds if the document expires shortly before the next monitor pass, but it can also be close to a minute and longer under load.

You can also insert a document with a timestamp that is already in the past. MongoDB removes it on the next TTL pass, not immediately.

      // TTL Index
      collection.createIndex(Indexes.ascending("date"),
          new IndexOptions().expireAfter(1L, TimeUnit.MINUTES));

      Document logMessage = new Document();
      logMessage.append("date", Date.from(Instant.now().minus(5, ChronoUnit.MINUTES)));
      logMessage.append("severity", "INFO");
      logMessage.append("message", "in the past");
      collection.insertOne(logMessage);

      System.out.println(collection.countDocuments()); // 1

      TimeUnit.MINUTES.sleep(1);

      System.out.println(collection.countDocuments()); // 0

Ttl2.java

Specific Expiry Date ¶

If each document needs its own expiration time, store that timestamp in a dedicated field such as expireAt and create the TTL index with an expireAfter value of 0.

      MongoCollection<Document> collection = db.getCollection("log");
      collection.createIndex(Indexes.ascending("expireAt"),
          new IndexOptions().expireAfter(0L, TimeUnit.SECONDS));

Ttl3.java

Now each document can carry its own exact expiration time.

      Document logMessage = new Document();
      logMessage.append("date", Date.from(Instant.now()));
      logMessage.append("expireAt", Date.from(Instant.now().plus(10, ChronoUnit.SECONDS)));
      logMessage.append("severity", "INFO");
      logMessage.append("message", "an info message");
      collection.insertOne(logMessage);

      logMessage = new Document();
      logMessage.append("date", Date.from(Instant.now()));
      logMessage.append("expireAt", Date.from(Instant.now().plus(2, ChronoUnit.MINUTES)));
      logMessage.append("severity", "WARN");
      logMessage.append("message", "a warning message");
      collection.insertOne(logMessage);

      logMessage = new Document();
      logMessage.append("date", Date.from(Instant.now()));
      logMessage.append("expireAt", Date.from(Instant.now().plus(5, ChronoUnit.MINUTES)));
      logMessage.append("severity", "ERROR");
      logMessage.append("message", "an error message");
      collection.insertOne(logMessage);

Ttl3.java

MongoDB removes each document after the time in expireAt has passed. There is still a monitor-cycle delay, so the removal time is near the requested timestamp, not exactly equal to it.

      System.out.println(collection.countDocuments()); // 3
      TimeUnit.SECONDS.sleep(60);
      System.out.println(collection.countDocuments()); // 2
      TimeUnit.SECONDS.sleep(120);
      System.out.println(collection.countDocuments()); // 1
      TimeUnit.SECONDS.sleep(180);
      System.out.println(collection.countDocuments()); // 0

Ttl3.java

Existing Indexes ¶

Starting with MongoDB 5.1, collMod can convert an existing non-TTL single-field index into a TTL index, or change the expireAfterSeconds value of an existing TTL index, without rebuilding the index from scratch.

Restrictions ¶

There are a few restrictions and operational details to keep in mind:

A TTL index must be a single-field index. If you create a compound index with expireAfter, MongoDB ignores the option.
The _id field does not support TTL indexes.
If a non-TTL single-field index already exists on the same key, use collMod instead of creating a second index with the same key pattern.
You cannot create a TTL index on a capped collection because capped collections do not support document removal.
On time series collections, MongoDB also supports collection-level expiration and, starting with MongoDB 7.0, partial TTL indexes on the metaField.

Schema validation is also a good companion for TTL indexes because it helps ensure the indexed field is always present and stored as a BSON date.

See the official documentation for a more in-depth description of TTL indexes. https://www.mongodb.com/docs/manual/core/index-ttl/

You can find all the code examples from this blog post here:
https://github.com/ralscha/blog/tree/master/capped