LJC Lightning Talk: "JaVers: Easy Audit Logs in Java"

Talk

The London Java Community (LJC) regularly hosts lightning talks. On Friday, October 2, 2020, I discussed how JaVers makes audit logs easy in Java.

I want to thank Dominique Carlo and Barry Cranford from RecWorks for making these lightning talks possible! If you need to hire Java developers in London, please consider RecWorks.

Video

My talk is on YouTube in the LJC channel:

Slides & Code

Grab the PDF slides here.

If you’re on an Apple device and have Apple’s “Keynote” presentation program installed, then you can also get the slides in their original Keynote format. Why would you do that? These slides are animated, so they are more pleasant to watch. Or maybe you want to peek under the hood to see how I achieved certain effects. Either way, follow this link to download these slides. You can view them with Apple’s “Keynote” program.

Getting Started

So you want to add an audit log to your application with JaVers? Wonderful! You’ve come to the right place!

Library & Configuration

The JaVers “Getting Started” documentation tells us how to add the library to your application. There are three ways to include JaVers:

• Spring Boot Starter
  That is by far the easiest way to get JaVers and what I used in my application. You have to pick a storage option for your versions: a relational database or MongoDB. As of October 1, 2020, the current Spring Boot Starter version 5.12.0 requires Spring Boot 2.3. If you still use Spring Boot 2.2, then you need to use JaVers version 5.9.4. I didn’t change any of the configuration options.
• Spring Integration
  If you use Spring without Spring Boot, then this is your option. I’ve never used this.
• Vanilla JaVers
  This works for all non-Spring environments. I haven’t used that, either.

The code below uses the current version of the Spring Boot Starter 5.12.0 with a relational database.

Create a Version

We recall that we version the DTOs that our business layer receives from and sends to the user interface. So let’s assume we have a customer DTO called Customer1:

public class Customer1 {

  private Long id;
  private String firstName;
  private String lastName;

Every object that JaVers versions needs a unique id. Fortunately, our DTOs already have that! So the easiest way is to mark the ID field in our DTO is the org.javers.core.metamodel.annotation.Id annotation in line 3:

public class Customer1 {

  @Id
  private Long id;
  private String firstName;
  private String lastName;

Now we’re ready to create a first version of this customer. Besides the DTO, all we need is a user name. We’ll use “joebloggs” in line 11:

  private Javers javers;

  [...]

  Customer1 customer = new Customer1(50l, "Karsten", "Silz");
  this.javers.commit("joebloggs", customer);

• We get the Javers instance through dependency injection (line 1).
• We create our DTO in line 5.
• The commit() call in line 6 converts the customer object into JSON and stores a version in the JaVers tables.

Congratulations — we created our first version with JaVers!

The JaVers Tables

JaVers uses 4 tables. We look at the two most important ones. The first one is JV_COMMIT. It stores one record per version. Here are the important columns:

• The COMMIT_PK is the globally unique ID of our object version: Across all objects and all versions, only one particular version has the COMMIT_PK of 100.
• AUTHOR is the user name that we passed into the JaVers commit() call above.
• COMMIT_DATE is when that version was created. JaVers sets this value for us.
• The COMMIT_ID is the locally unique ID version number of our object: Only one version of our customer with the ID 50 has that COMMIT_ID.

JV_SNAPSHOT stores the JSON data:

• The SNAPSHOT_PK is the COMMIT_PK from the JV_COMMIT table above.
• The TYPE is INITIAL because this is the first version. Later versions have the type UPDATE.
• STATE is our Customer1 instance from above as JSON.
• CHANGED_PROPERTIES contains which fields of our object have changed compared to the last version. Since this is the first version, all fields have.
• MANAGED_TYPE is the full name of our class.

Query for All Versions

Now that we created a version with JaVers. How do query versions? JaVers gives us a query language, using the so-called fluid notation. Let’s retrieve all versions of our customer DTO first. We could show the result in a list or table to our users:

    JqlQuery allVersionsQuery =
        QueryBuilder.byInstanceId(50l, Customer1.class)
            .withShadowScope(ShadowScope.SHALLOW)
            .build();
    List<CdoSnapshot> versions = this.javers.findSnapshots(allVersionsQuery);

    for (CdoSnapshot aVersion : versions) {
      String versionNumber = aVersion.getCommitId().value();
      CommitMetadata commitMetadata = aVersion.getCommitMetadata();
      String author = commitMetadata.getAuthor();
      LocalDateTime time = commitMetadata.getCommitDate();
      System.out.println(
          "Version details: versionNumber=" + versionNumber
              + ", author=" + author
              + ", time=" + time);
    }

Here’s the result output with just one version:

Version details: versionNumber=1.00, author=joebloggs, time=2020-10-01T18:46:27.567

So what did we do here?

• We restrict the query to the version of the Customer1 class with the ID of 50 (line 2).
• Our objects could be nested and contain other objects. We’re just interested in an overview here, we don’t want to retrieve these nested objects. That’s why we specify the ShadowScope.SHALLOW (line 3).
• The querying happens in line 5. We’re retrieving what JaVers calls a snapshot. It contains our versioned object as a property map. Since we don’t want to access the versioned object here, that’s fine with us.
• JaVers returns a list of CdoSnapshot (line 5). These contain both the version information and the actual objects.
• The version number is the so-called commit ID — COMMIT_ID from the JV_COMMIT (line 8). Although it’s a 1.0 in the database, getCommitId().value() returns 1.00. In my experience, you can ignore the decimal places since the next versions would be 2.0, 3.0, and so forth.
• We get the author of the version (line 10) and the time it was created (line 11).

Query For One Version

We saw in the previous section how we can retrieve all versions of our DTOs. How do we get a single version then? Well, nearly the same way:

    CommitId commitId = CommitId.valueOf("1.0");
    JqlQuery oneVersionQuery =
        QueryBuilder.byInstanceId(50l, Customer1.class)
            .withCommitId(commitId)
            .withShadowScope(ShadowScope.DEEP_PLUS)
            .build();
    List<Shadow<Customer1>> singleVersion = this.javers.findShadows(oneVersionQuery);

    for (Shadow<Customer1> aVersion : singleVersion) {
      String versionNumber = aVersion.getCommitId().value();
      CommitMetadata commitMetadata = aVersion.getCommitMetadata();
      String author = commitMetadata.getAuthor();
      LocalDateTime time = commitMetadata.getCommitDate();
      Customer1 customer = aVersion.get();
      System.out.println(
          "Version details: versionNumber=" + versionNumber
              + ", author=" + author
              + ", time=" + time
              + ", customer=" + customer);
    }

Here’s the output:

Version details: versionNumber=1.00, author=joebloggs, time=2020-10-01T18:46:27.567, customer=Customer1{id=50, firstName='Karsten', lastName='Silz'}

Here’s what we did differently to get one version:

• We specify the commit ID 1.0 (lines 1 and 4).
• We tell JaVers to rebuild nested objects with the ShadowScope.DEEP_PLUS (line 5). Our Customer1 class doesn’t have nested objects, but objects in our real applications probably do.
• We now query for so-called Shadows (lines 7 and 9). They do include our versioned object (line 14).

Comparing Versions

The most complex operation is to compare two objects with each other. Why? Because there can be many different changes, all expressed as their own classes in JaVers. The comparison of two objects itself is simple. In the example, below, both the first name and the last name change:

    Customer1 oldOne = new Customer1(50l, "Karsten", "Silz");
    Customer1 newOne = new Customer1(50l, "Joe", "Cool");
    Diff differences = this.javers.compare(oldOne, newOne);
    System.out.println("Differences: " + differences.prettyPrint());

Predictably, JaVers found both changes:

Differences: Diff:
* changes on com.betterprojectsfaster.talks.javers.customer1.Customer1/50 :
  - 'firstName' value changed from 'Karsten' to 'Joe'
  - 'lastName' value changed from 'Silz' to 'Cool'

Now in our applications, we need the details of the changes. Here’s how we get them:

    Customer1 oldOne = new Customer1(50l, "Karsten", "Silz");
    Customer1 newOne = new Customer1(50l, "Joe", "Cool");
    Diff differences = this.javers.compare(oldOne, newOne);
    Changes listOfDifferences = differences.getChanges();

    for (Change aChange : listOfDifferences) {
      System.out.println("  Change: " + aChange.toString());
    }

This is the output:

Change: ValueChange{ 'firstName' value changed from 'Karsten' to 'Joe' }
Change: ValueChange{ 'lastName' value changed from 'Silz' to 'Cool' }

Now the sample, we just got a single change — ValueChange instances. So here are some of the changes we will encounter in our work, each expressed as a different JaVers class:

• ValueChange
  This indicates a single field change. You get the field name with getPropertyName(), the old value with getLeft(), and the new one with getRight().
• NewObject
  The new version contains a completely new object. We can get the class name and ID of the new object with getAffectedGlobalId().
• ObjectRemoved
  The new version does not contain a certain object anymore. We also can get the class name and ID of the new object with getAffectedGlobalId().
• ListChange
  One or more changes happened in a list of the new version. getChanges() returns these changes as a List<ContainerElementChange>. These ContainerElementChange can be:
  • ElementValueChange shows an element has changed. getIndex() gives us the position in the list, getLeftValue() the old value, getRightVlue() the new one.
  • ValueAdded is a new element in the new version. getIndex() gives us the position in the list, getAddedValue() the added value.
  • ValueRemoved is a deleted element in the new version. getIndex() gives us the position in the list, getRemovedValue() the removed value.

Nested Objects

Our classes often include other classes. So let’s extend our customer DTO to include a contact information DTO which itself has a list of email address DTOs:

public enum EmailType { PRIVATE, WORK }

public class EmailAddress {
  @Id private final Long id;
  private final EmailType type;
  private final String address;

  [...]
}

public class ContactInfo {
  @Id private final Long id;
  private final List<EmailAddress> emailAddresses;


  [...]
}

public class Customer2 {

  @Id private final Long id;
  private final String firstName;
  private final String lastName;
  private final ContactInfo contactInfo;

  [...]
}

• Our new customer DTO class is called Customer2. It has a contactInfo field (line 24).
• The ContactInfo class has a list of EmailAddress instances (line 13).
• EmailAddress has a type field (line 5) and the address (line 6).

So let’s create a version with nested objects:

EmailAddress address1 = new EmailAddress(20l, EmailType.PRIVATE, "private@provider.com");
EmailAddress address2 = new EmailAddress(21l, EmailType.WORK, "work@provider.com");
ContactInfo info = new ContactInfo(63l, List.of(address1, address2));
Customer2 myCustomer = new Customer2(50l, "Karsten", "Silz", info);
this.javers.commit("runner1", myCustomer);

What does that produce in the JV_SNAPSHOT table? Since each nested object has an ID field, we expect each one to get a separate entry. And they do:

• The two email addresses occupy the first two records (SNAPSHOT_PK of 100 and 200).
• The customer is the third row SNAPSHOT_PK of 300). It includes the contact information by reference:

“contactInfo”: { “entity”: “com.company.product. dto.ContactInfo”, “cdoId”: 63 }

• The referenced contact information is record number four (SNAPSHOT_PK of 400). It contains the two email addresses as references, too:

{ “emailAddresses”: [ 
  { “entity”: “com.company.product. dto.EmailAddress”, “cdoId”: 20 }, 
  { “entity”: “com.company.product. dto.EmailAddress”, “cdoId”: 21 }
]

Now, if we retrieve an instance of Customer2 with JaVers, do we get all the nested objects, too?

    CommitId commitId = CommitId.valueOf("1.0");
    JqlQuery oneVersionQuery =
        QueryBuilder.byInstanceId(50l, Customer2.class)
            .withCommitId(commitId)
            .withShadowScope(ShadowScope.DEEP_PLUS)
            .build();
    List<Shadow<Customer2>> singleVersion = this.javers.findShadows(oneVersionQuery);

    for (Shadow<Customer2> aVersion : singleVersion) {
      Customer2 customer = aVersion.get();
      System.out.println("Customer=" + customer);
    }

Yes, we do (formatted for display):

Customer: Customer2{id=50, firstName='Karsten', lastName='Silz', 
  contactInfo=ContactInfo{id=63, emailAddresses=[
    EmailAddress{id=20, type=PRIVATE, address='private@provider.com'}, 
    EmailAddress{id=21, type=WORK, address='work@provider.com'}
  ]}
}

Customizing Versioning

In this last section, we’ll do some meaningful customizations. Let’s recall the first class we versioned:

public class Customer1 {

  @Id
  private Long id;
  private String firstName;
  private String lastName;

And it produced this version entry in JV_SNAPSHOT:

The JSON data is quite verbose. Since we’ll store a lot of versions in our real applications, shrinking that JSON size will save database space. So, what can we do here?

I actually wrote an article at Baeldung on how to do just that. I wonder what inspired me here?  🤔   The one technique that applies here is to use short field names. So instead of "firstName": "Karsten", we’ll get "f": "Karsten". In the article, this reduced the JSON data size by more than 25%. Since the field names also appear in the CHANGED_PROPERTIES column, we can reduce the database space that our versions take up!

So let’s apply short field names in our new DTO class Customer3. JaVers gives us the org.javers.core.metamodel.annotation.PropertyName annotation to specify the field name in the database (lines 4, 6 and 8):

public class Customer3 {

  @Id
  @PropertyName("i")
  private Long id;
  @PropertyName("f")
  private String firstName;
  @PropertyName("l")
  private String lastName;

This results in the following entry in JV_SNAPSHOT:

As we can see, both STATE and CHANGED_PROPERTIES use the one-letter field names. Nice savings!

Beyond the space savings, refactoring of our Java classes down the road is another reason for customized field names. Let’s imagine we rename lastName to familyName in our Java class. This would break the retrieval of old versions: The JSON in the database has lastName for the old versions and familyName for the new ones. But with the @Field("l") annotation, all versions store the field name as l in JSON.

One thing still sticks out in the database entry above: The value in the MANAGED_TYPE column is the fully qualified class name com.company.product.dto.Customer3. There’s an annotation for that, too — org.javers.core.metamodel.annotation.TypeName(line 1):

@TypeName("C3")
public class Customer3 {

  @Id
  @Field("i")
  private Long id;
  @Field("f")
  private String firstName;
  @Field("l")
  private String lastName;

Now we’re talking:

That’s as compact as it’s going to get!

Another customization is to exclude fields from versioning. Let’s say that we add the number of versions to the class itself. We calculate that by querying the versions before we send the object to our front-end. Now we don’t want this field in our version data. Lucky for us, the org.javers.core.metamodel.annotation.DiffIgnore annotation lets JaVers ignore fields (line 11):

@Class("C2")
public class Customer2 {

  @Id
  @Field("i")
  private Long id;
  @Field("f")
  private String firstName;
  @Field("l")
  private String lastName;
  @DiffIgnore
  private Integer numberOfVersions;

Wrap-Up

This concludes our code introduction. Let’s recap:

• We started with how to add JaVers to our projects.
• We took a class and created a version of it.
• Then we queried JaVers for all versions and for a single version.
• Next, we compared two instances of a class.
• Nested objects and their versioning was the next stop on our tour.
• Finally, we looked at customizing the versioning.

So head over to JaVers and put a cool audit log into your Java application!