LJC Community Talk: "JaVers: Code Audit Logs Easily in Java"
Talk
The London Java Community (LJC) regularly hosted community talks, where two speakers each got 30 minutes. On Wednesday, October 21, 2020, I discussed there how JaVers makes audit logs easy in Java.
This was an extended version of the lightning talk I gave nearly three weeks earlier. Head over there for a quick introduction to audit logs with JaVers!
I want to thank Dominique Carlo and Barry Cranford from RecWorks. They made these community talks possible! So if you need to hire Java developers in London, please consider RecWorks.
Video
The talk is on YouTube. It’s 26 minutes and 33 seconds long:
Slides & Code
The slides are available as a PDF. Because I included selected animation changes, it’s 97 pages:
Historical note (as of October 2020): Back then, the slides were also available in their original Keynote format with animations; that file is no longer available today.
Getting Started
So you want to add an audit log to your application with JaVers? Wonderful! You’ve come to the right place!
Library & Configuration
The JaVers “Getting Started” documentation tells us how to add the library to your application. There are three ways to include JaVers:
- Spring Boot Starter — by far the easiest way to get JaVers, and what I used in my application. You have to pick a storage option for your versions: a relational database or MongoDB. As of October 21, 2020, the current Spring Boot Starter version 5.13.0 required Spring Boot 2.3. If you still used Spring Boot 2.2, you needed JaVers version 5.9.4. I didn’t change any of the configuration options.
- Spring Integration — the option for Spring without Spring Boot. I never used this.
- Vanilla JaVers — works for all non-Spring environments. I didn’t use this either.
The code below uses the then-current version of the Spring Boot Starter, 5.13.0, with a relational database.
Create a Version
We recall that we version the DTOs that our business layer receives from and sends to the user interface. So let’s assume we have a customer DTO called Customer1:
public class Customer1 {
private Long id;
private String firstName;
private String lastName;
Every object that JaVers versions needs a unique ID. Fortunately, our DTOs already have that! So the easiest way is to mark the ID field in our DTO with the org.javers.core.metamodel.annotation.Id annotation, in line 3:
public class Customer1 {
@Id
private Long id;
private String firstName;
private String lastName;
Now we’re ready to create a first version of this customer. Besides the DTO, all we need is a user name. We’ll use “joebloggs”:
private Javers javers;
[...]
Customer1 customer = new Customer1(50L, "Karsten", "Silz");
this.javers.commit("joebloggs", customer);
- We get the
Javersinstance through dependency injection. - We create our DTO.
- The
commit()call converts thecustomerobject into JSON and stores a version in the JaVers tables.
Congratulations — we created our first version with JaVers!
The JaVers Tables
JaVers uses four tables. We’ll look at the two most important ones. The first one is JV_COMMIT. It stores one record per version. Here are the important columns:
| COMMIT_PK | AUTHOR | COMMIT_DATE | COMMIT_ID |
|---|---|---|---|
| 100 | joebloggs | 2020-10-01 18:46:27.567 | 1.0 |
- The
COMMIT_PKis the globally unique ID of our object version: across all objects and all versions, only one particular version has theCOMMIT_PKof100. AUTHORis the user name we passed into the JaVerscommit()call above.COMMIT_DATEis when that version was created. JaVers sets this value for us.- The
COMMIT_IDis the locally unique version number of our object: only one version of our customer with the ID50has thatCOMMIT_ID.
JV_SNAPSHOT stores the JSON data:
| SNAPSHOT_PK | TYPE | STATE | CHANGED_PROPERTIES | MANAGED_TYPE |
|---|---|---|---|---|
| 100 | INITIAL | { "firstName": "Karsten", "lastName": "Silz", "id": 50 } |
["firstName", "lastName", "id"] |
com.company.product.dto.Customer1 |
- The
SNAPSHOT_PKis theCOMMIT_PKfrom theJV_COMMITtable above. - The
TYPEisINITIALbecause this is the first version. Later versions have the typeUPDATE. STATEis ourCustomer1instance from above, as JSON.CHANGED_PROPERTIESlists which fields of our object changed compared to the last version. Since this is the first version, all fields did.MANAGED_TYPEis the full name of our class.
Query for All Versions
Now that we created a version with JaVers, how do we query versions? JaVers gives us a query language using the so-called fluent notation. Let’s retrieve all versions of our customer DTO first. We could show the result in a list or table to our users:
JqlQuery allVersionsQuery =
QueryBuilder.byInstanceId(50L, Customer1.class)
.withShadowScope(ShadowScope.SHALLOW)
.build();
List<CdoSnapshot> versions = this.javers.findSnapshots(allVersionsQuery);
for (CdoSnapshot aVersion : versions) {
String versionNumber = aVersion.getCommitId().value();
CommitMetadata commitMetadata = aVersion.getCommitMetadata();
String author = commitMetadata.getAuthor();
LocalDateTime time = commitMetadata.getCommitDate();
System.out.println(
"Version details: versionNumber=" + versionNumber
+ ", author=" + author
+ ", time=" + time);
}
Here’s the result output with just one version:
Version details: versionNumber=1.00, author=joebloggs, time=2020-10-01T18:46:27.567
So what did we do here?
- We restricted the query to the version of the
Customer1class with the ID of50. - Our objects could be nested and contain other objects. We were just interested in an overview here, so we didn’t want to retrieve these nested objects. That’s why we specified
ShadowScope.SHALLOW. - The query itself retrieves what JaVers calls a snapshot. It contains our versioned object as a property map. Since we didn’t need to access the versioned object here, that was fine.
- JaVers returns a list of
CdoSnapshot. These contain both the version information and the actual objects. - The version number is the so-called commit ID —
COMMIT_IDfromJV_COMMIT. Although it’s1.0in the database,getCommitId().value()returns1.00. In my experience, you can ignore the decimal places, since the next versions would be2.0,3.0, and so forth. - We get the author of the version and the time it was created.
Query for One Version
We saw in the previous section how to retrieve all versions of our DTOs. How do we get a single version? Nearly the same way:
CommitId commitId = CommitId.valueOf("1.0");
JqlQuery oneVersionQuery =
QueryBuilder.byInstanceId(50L, Customer1.class)
.withCommitId(commitId)
.withShadowScope(ShadowScope.DEEP_PLUS)
.build();
List<Shadow<Customer1>> singleVersion = this.javers.findShadows(oneVersionQuery);
for (Shadow<Customer1> aVersion : singleVersion) {
String versionNumber = aVersion.getCommitId().value();
CommitMetadata commitMetadata = aVersion.getCommitMetadata();
String author = commitMetadata.getAuthor();
LocalDateTime time = commitMetadata.getCommitDate();
Customer1 customer = aVersion.get();
System.out.println(
"Version details: versionNumber=" + versionNumber
+ ", author=" + author
+ ", time=" + time
+ ", customer=" + customer);
}
Here’s the output:
Version details: versionNumber=1.00, author=joebloggs, time=2020-10-01T18:46:27.567, customer=Customer1{id=50, firstName='Karsten', lastName='Silz'}
Here’s what we did differently to get one version:
- We specified the commit ID
1.0. - We told JaVers to rebuild nested objects with
ShadowScope.DEEP_PLUS. OurCustomer1class didn’t have nested objects, but objects in real applications probably do. - We now queried for so-called Shadows. They do include our versioned object.
Comparing Versions
The most complex operation is comparing two objects with each other. Why? Because there can be many different kinds of changes, each expressed as its own class in JaVers. Comparing two objects itself is simple. In the example below, both the first name and the last name change:
Customer1 oldOne = new Customer1(50L, "Karsten", "Silz");
Customer1 newOne = new Customer1(50L, "Joe", "Cool");
Diff differences = this.javers.compare(oldOne, newOne);
System.out.println("Differences: " + differences.prettyPrint());
Predictably, JaVers found both changes:
Differences: Diff:
* changes on com.betterprojectsfaster.talks.javers.customer1.Customer1/50 :
- 'firstName' value changed from 'Karsten' to 'Joe'
- 'lastName' value changed from 'Silz' to 'Cool'
In our applications, we usually need the details of the changes. Here’s how we get them:
Customer1 oldOne = new Customer1(50L, "Karsten", "Silz");
Customer1 newOne = new Customer1(50L, "Joe", "Cool");
Diff differences = this.javers.compare(oldOne, newOne);
Changes listOfDifferences = differences.getChanges();
for (Change aChange : listOfDifferences) {
System.out.println(" Change: " + aChange.toString());
}
This is the output:
Change: ValueChange{ 'firstName' value changed from 'Karsten' to 'Joe' }
Change: ValueChange{ 'lastName' value changed from 'Silz' to 'Cool' }
In this sample, we only got a single kind of change — ValueChange instances. Here are some of the changes you’ll encounter in your own work, each expressed as a different JaVers class:
ValueChange— indicates a single field change. You get the field name withgetPropertyName(), the old value withgetLeft(), and the new one withgetRight().NewObject— the new version contains a completely new object. You can get the class name and ID of the new object withgetAffectedGlobalId().ObjectRemoved— the new version no longer contains a certain object. You can also get the class name and ID withgetAffectedGlobalId().ListChange— one or more changes happened in a list in the new version.getChanges()returns these as aList<ContainerElementChange>. TheseContainerElementChangeinstances can be:ElementValueChange— an element changed.getIndex()gives the position in the list,getLeftValue()the old value,getRightValue()the new one.ValueAdded— a new element in the new version.getIndex()gives the position,getAddedValue()the added value.ValueRemoved— a deleted element in the new version.getIndex()gives the position,getRemovedValue()the removed value.
Counting Changes
We saw in the previous section that JaVers returns changes when comparing objects. But the number of changes JaVers reports can differ from the number our users expect!
An example: our Customer1 class has both a firstName and a lastName. If both change, that’s two changes to JaVers. But if our UI shows both in one “Name” field, it’s just one change to our users.
The solution to this problem is to customize how we count the changes for these objects. It’s probably best to do that in the UI, since the display dictates how many changes the user perceives. For instance, if the UI suddenly displays first name and last name separately, we’d be back to two changes!
Versioning Nested Objects
Our classes often include other classes. So let’s extend our customer DTO to include a contact information DTO, which itself has a list of email address DTOs:
public enum EmailType { PRIVATE, WORK }
public class EmailAddress {
@Id private final Long id;
private final EmailType type;
private final String address;
[...]
}
public class ContactInfo {
@Id private final Long id;
private final List<EmailAddress> emailAddresses;
[...]
}
public class Customer2 {
@Id private final Long id;
private final String firstName;
private final String lastName;
private final ContactInfo contactInfo;
[...]
}
- Our new customer DTO class is called
Customer2. It has acontactInfofield. - The
ContactInfoclass has a list ofEmailAddressinstances. EmailAddresshas atypefield and anaddressfield.
So let’s create a version with nested objects:
EmailAddress address1 = new EmailAddress(20L, EmailType.PRIVATE, "private@provider.com");
EmailAddress address2 = new EmailAddress(21L, EmailType.WORK, "work@provider.com");
ContactInfo info = new ContactInfo(63L, List.of(address1, address2));
Customer2 myCustomer = new Customer2(50L, "Karsten", "Silz", info);
this.javers.commit("runner1", myCustomer);
What does that produce in the JV_SNAPSHOT table? Since each nested object has an ID field, we’d expect each one to get a separate entry. And they do:
| SNAPSHOT_PK | TYPE | STATE | CHANGED_PROPERTIES | MANAGED_TYPE |
|---|---|---|---|---|
| 100 | INITIAL | { "address": "work@provider.com", "id": 21, "type": "WORK" } |
["address", "id", "type"] |
com.company.product.dto.EmailAddress |
| 200 | INITIAL | { "address": "private@provider.com", "id": 20, "type": "PRIVATE" } |
["address", "id", "type"] |
com.company.product.dto.EmailAddress |
| 300 | INITIAL | { "firstName": "Karsten", "lastName": "Silz", "contactInfo": { "entity": "com.company.product.dto.ContactInfo", "cdoId": 63 }, "id": 50 } |
["firstName", "lastName", "contactInfo", "id"] |
com.company.product.dto.Customer2 |
| 400 | INITIAL | { "emailAddresses": [ { "entity": "com.company.product.dto.EmailAddress", "cdoId": 20 }, { "entity": "com.company.product.dto.EmailAddress", "cdoId": 21 } ], "id": 63 } |
["emailAddresses", "id"] |
com.company.product.dto.ContactInfo |
- The two email addresses occupy the first two records (
SNAPSHOT_PK100 and 200). - The customer is the third row (
SNAPSHOT_PK300). It includes the contact information by reference:
"contactInfo": { "entity": "com.company.product.dto.ContactInfo", "cdoId": 63 }
- The referenced contact information is record number four (
SNAPSHOT_PK400). It contains the two email addresses as references, too:
{ "emailAddresses": [
{ "entity": "com.company.product.dto.EmailAddress", "cdoId": 20 },
{ "entity": "com.company.product.dto.EmailAddress", "cdoId": 21 }
] }
Now, if we retrieve an instance of Customer2 with JaVers, do we get all the nested objects, too?
CommitId commitId = CommitId.valueOf("1.0");
JqlQuery oneVersionQuery =
QueryBuilder.byInstanceId(50L, Customer2.class)
.withCommitId(commitId)
.withShadowScope(ShadowScope.DEEP_PLUS)
.build();
List<Shadow<Customer2>> singleVersion = this.javers.findShadows(oneVersionQuery);
for (Shadow<Customer2> aVersion : singleVersion) {
Customer2 customer = aVersion.get();
System.out.println("Customer=" + customer);
}
Yes, we do (formatted for display):
Customer: Customer2{id=50, firstName='Karsten', lastName='Silz',
contactInfo=ContactInfo{id=63, emailAddresses=[
EmailAddress{id=20, type=PRIVATE, address='private@provider.com'},
EmailAddress{id=21, type=WORK, address='work@provider.com'}
]}
}
Querying Nested Objects
We just saw that we can version objects that contain other objects. There is one problem, though.
Say we update one of the email addresses that now belong to Customer2. That instance of EmailAddress gets a new version number. But the Customer2 instance does not! When a nested object gets a new version number, the parent object doesn’t automatically get one, too.
From JaVers’ point of view, that’s correct: the customer didn’t change, an email address did. But from an application user’s point of view, it’s wrong: the email belongs to the customer, so the customer should have a new version, too!
How do we solve this?
First of all, that’s a feature, not a bug. And JaVers isn’t alone here: Hibernate Envers behaves exactly the same. There are different solutions. Here’s mine: I make sure the parent object (Customer2 here) always changes when creating a version. This way, the parent object always gets a new version whenever one of its nested objects changes.
More specifically, I add a boolean field that flips between true and false. This takes up less space in the database than, say, a long random number. The downside is that I need to query the current version from JaVers before I can save a new one.
Customizing Versioning
In this last section, we’ll make some meaningful customizations. Let’s recall the first class we versioned:
public class Customer1 {
@Id
private Long id;
private String firstName;
private String lastName;
It produced this version entry in JV_SNAPSHOT:
| SNAPSHOT_PK | TYPE | STATE | CHANGED_PROPERTIES | MANAGED_TYPE |
|---|---|---|---|---|
| 100 | INITIAL | { "firstName": "Karsten", "lastName": "Silz", "id": 50 } |
["firstName", "lastName", "id"] |
com.company.product.dto.Customer1 |
The JSON data is quite verbose. Since we’d store a lot of versions in real applications, shrinking that JSON size saves database space. So what can we do?
I actually wrote an article at Baeldung on how to do just that. I wonder what inspired me there? 🤔 The one technique that applies here is using short field names. So instead of "firstName": "Karsten", we get "f": "Karsten". In the article, this reduced the JSON data size by more than 25%. Since the field names also appear in the CHANGED_PROPERTIES column, this reduces the database space our versions take up, too!
So let’s apply short field names in our new DTO class Customer3. JaVers gives us the org.javers.core.metamodel.annotation.PropertyName annotation to specify the field name in the database:
public class Customer3 {
@Id
@PropertyName("i")
private Long id;
@PropertyName("f")
private String firstName;
@PropertyName("l")
private String lastName;
This results in the following entry in JV_SNAPSHOT:
| SNAPSHOT_PK | TYPE | STATE | CHANGED_PROPERTIES | MANAGED_TYPE |
|---|---|---|---|---|
| 101 | INITIAL | { "f": "Karsten", "l": "Silz", "i": 50 } |
["f", "l", "i"] |
com.company.product.dto.Customer3 |
As you can see, both STATE and CHANGED_PROPERTIES use the one-letter field names. Nice savings!
Beyond the space savings, refactoring our Java classes down the road is another reason for customized field names. Imagine we rename lastName to familyName in our Java class. That would break the retrieval of old versions: the JSON in the database has lastName for the old versions and familyName for the new ones. But with the @PropertyName("l") annotation, all versions store the field name as l in JSON.
JaVers automatically translates between the short, saved property name (such as f) and the long one in Java (such as firstName) when retrieving objects. Great! But it does not do that when giving us a list of changed properties — there, we get the short ones (such as f). How do we deal with that?
There are multiple ways to handle this. I created a Map with both the short and the long names:
private static final Map<String, String> NAME_MAP_CUSTOMER = Stream
.of(
new String[][] {
{ "i", "id" },
{ "f", "firstName" },
{ "l", "lastName" },
}
)
.collect(Collectors.collectingAndThen(Collectors.toMap(data -> data[0], data -> data[1]),
Collections::unmodifiableMap));
This way of initializing a static map looks complicated. But it has the advantage that it works with more than ten fields, unlike Map.of().
I then use this map to look up the long name of a property:
String lookUpLongName(String shortName) {
return NAME_MAP_CUSTOMER.get(shortName);
}
One thing still stands out in the database entry above: the value in the MANAGED_TYPE column is the fully qualified class name com.company.product.dto.Customer3. There’s an annotation for that, too — org.javers.core.metamodel.annotation.TypeName:
@TypeName("C3")
public class Customer3 {
@Id
@PropertyName("i")
private Long id;
@PropertyName("f")
private String firstName;
@PropertyName("l")
private String lastName;
Now we’re talking:
| SNAPSHOT_PK | TYPE | STATE | CHANGED_PROPERTIES | MANAGED_TYPE |
|---|---|---|---|---|
| 101 | INITIAL | { "f": "Karsten", "l": "Silz", "i": 50 } |
["f", "l", "i"] |
C3 |
That’s as compact as it gets!
Another customization is excluding fields from versioning. Say we add the number of versions to the class itself, calculated by querying the versions before we send the object to our front-end. We don’t want this field in our version data. Lucky for us, the org.javers.core.metamodel.annotation.DiffIgnore annotation lets JaVers ignore fields:
@TypeName("C2")
public class Customer2 {
@Id
@PropertyName("i")
private Long id;
@PropertyName("f")
private String firstName;
@PropertyName("l")
private String lastName;
@DiffIgnore
private Integer numberOfVersions;
Wrap-Up
This concludes our code introduction. Let’s recap:
- We started with how to add JaVers to our projects.
- We took a class and created a version of it.
- Then we queried JaVers for all versions and for a single version.
- Next, we compared two instances of a class.
- Nested objects and their versioning were the next stop on our tour.
- Finally, we looked at customizing the versioning.
Head over to JaVers and put a great audit log into your Java application!