2.9.4 doesn’t handle `@JsonIgnore`d field with the same name as `@JsonProperty(name)` of other field

[silmeth]

I have a data class that looks something like this:

data class DTO(@JsonProperty("name") val name: String,
               @JsonIgnore val query: NonSerializableClass,
               @JsonProperty("query") val rawQuery: String) {
    @JsonCreator
    constructor(@JsonProperty("name") name: String,
                @JsonProperty("query") rawQuery: String) :
        this(name, parseNonSerializableField(rawQuery), rawQuery)
}

Where NonSerializableClass is a complicated external class that I do not want to write custom (de)serializers for, and whose instances can be easily created by parsing the rawQuery string.

With Jackson 2.9.3 and jackson-module-kotlin 2.9.3 string of the kind {"name": "foo", "query": "bar"} get nicely serialized into DTO and the object created can be nicely deserialized.

But with Jackson + jackson-module-kotlin 2.9.4 I get a com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException "due to missing (therefore NULL) value for creator parameter rawQuery which is a non-nullable type" (but after a bit of fiddling with it, I found out that the real problem is that there is a @JsonIgnore val query – with the same name as a @JsonProperty("name") of another field).

A minimal test for this, that passes with version 2.9.3, but fails with 2.9.4, looks like this:

import com.fasterxml.jackson.annotation.JsonCreator
import com.fasterxml.jackson.annotation.JsonIgnore
import com.fasterxml.jackson.annotation.JsonProperty
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.kotlin.KotlinModule
import com.fasterxml.jackson.module.kotlin.readValue
import org.junit.jupiter.api.Test

internal class DeserializationTest {
    @Test
    fun test() {
        val objMapper = ObjectMapper().registerModule(KotlinModule())

        // with 2.9.3 prints
        //     Foo(name=foo, query=NonSerializable, rawQuery=bar)
        // but with 2.9.4 throws:
        //     com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class DeserializationTest$Foo] value failed for JSON property query due to missing (therefore NULL) value for creator parameter rawQuery which is a non-nullable type
        //      at [Source: (String)"{"name": "foo", "query": "bar"}"; line: 1, column: 31] (through reference chain: DeserializationTest$Foo["query"])
        val deserialized: Foo = objMapper.readValue("{\"name\": \"foo\", \"query\": \"bar\"}")
        println(deserialized)

        val serialized = objMapper.writeValueAsString(deserialized)

        // with 2.9.3 prints
        //     {"name":"foo","query":"bar"}
        println(serialized)
    }

    class NonSerializable(private val field: Any?) {
        override fun toString() = "NonSerializable"
    }

    data class Foo(@JsonProperty("name") val name: String,
                   @JsonIgnore val query: NonSerializable,
                   @JsonProperty("query") val rawQuery: String) {
        @JsonCreator
        constructor(@JsonProperty("name") name: String,
                    @JsonProperty("query") rawQuery: String): this(name, NonSerializable(rawQuery), rawQuery)
    }
}

After changing the name of the non-serializable @JsonIgnored field query to something else, like ignoredQuery, it works in 2.9.4 too.

But that still looks like a bug, and I’d like not to change the API of my class (where value query returns the non-serializable object) nor to force all places where this data class is serialized to change its JSON schema (to use another field like eg. rawQuery instead of query).

[cowtowncoder]

I would need a Java-only reproduction here, if possible to reproduce in plain Java.
It not this needs to be filed at Kotlin module.

[silmeth]

This is the jackson-module-kotlin repository’s issue. Do you mean some other place?

BTW I did not try to reproduce this in Java without a Kotlin module, might try later.

[cowtowncoder]

@silmeth Oh. I'm sorry, somehow I misread URL. Never mind. :-)
(going through issue reports so sometimes get mixed)
Java repro would be useful but this issue is fine here.

[silmeth]

I tried this in Java, both through annotating accordingly public fields and by providing annotated getters and could not reproduce the wrong behaviour, so I am pretty sure this is a Kotlin-specific issue.

[apatrida]

@silmeth and @cowtowncoder Java won't call a custom value instantiator that checks nullability and missing parameter likely. What happens is that the ValueInstantiator is called with a list of CreatorProperty where one doesn't exist, rawQuery (since it's JsonProperty is set to query)

The way to reproduce this in Java would be to do a similar setup, create a custom ValueInstantiator and see what it receives. @silmeth can you check that?

[jskierbi]

I've debugged a bit.

Seems the test @silmeth provided is broken by this particular commit: FasterXML/jackson-databind@562484c

Change in this class: PropertyBasedCreator.java

query property is not added to _propertyLookup although defined in creator: @JsonProperty("query") rawQuery: String

This makes BeanDeserializer#_deserializeUsingPropertyBased skip query param from bufferring, and in turn results in KotlinValueInstantiator fail due to isMissing flag set to true (!buffer.hasParameter(jsonProp))

[jskierbi]

I've tried making sense of POJOPropertiesCollector, but no luck finding way to fix situation for Kotlin. I've managed, however, to get the exact same error with Java class.

First minimal Kotlin class triggering the error:

  data class FooKt(@JsonProperty("query") val rawQuery: String) {
    @JsonIgnore val query: String = rawQuery.trim()
  }

Then Java class that triggers similar logic inside POJOPropertiesCollector:

public class FooJava {

  @JsonIgnore // Mimics Kotlin ignored property that matches ctor parameter by name
  public String getQuery() {
    return "";
  }

  @JsonCreator
  @ConstructorProperties("rawQuery") // Mimics defined implicit name for Kotlin ctors
  public FooJava(@JsonProperty("query") @NotNull String rawQuery) {
  }
}

And the tests:

@Test
fun `deserialize kt`() {
  val deserialized: FooKt = jacksonObjectMapper().readValue("{\"query\": \"bar\"}")
  // 2.9.3: passes
  // 2.9.4: exception
  // com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class com.fasterxml.jackson.module.kotlin.test.Github124$FooKt] value failed for JSON property query due to missing (therefore NULL) value for creator parameter rawQuery which is a non-nullable type
  // at [Source: (String)"{"query": "bar"}"; line: 1, column: 16] (through reference chain: com.fasterxml.jackson.module.kotlin.test.Github124$FooKt["query"])
}

@Test
fun `deserialize java`() {
  val deserialized: FooJava = jacksonObjectMapper().readValue("{\"query\": \"bar\"}")
  // 2.9.3: passes
  // 2.9.4: exception
  // com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `com.fasterxml.jackson.module.kotlin.test.FooJava`, problem: Argument for @NotNull parameter 'rawQuery' of com/fasterxml/jackson/module/kotlin/test/FooJava.<init> must not be null
  // at [Source: (String)"{"query": "bar"}"; line: 1, column: 16]
}

Both above tests pass for 2.9.3 and fail for 2.9.4. Exceptoins raised for 2.9.4 are different, but they have the same root cause (BeanDeserializer#_deserializeUsingPropertyBased skips query param from json in buffer). Fixing root cause should fix both cases. Changing getQuery() in FooJava to e.g. getQueryX() makes Java test pass, which is aligned with Kotlin behaviour.

Any thoughts?

[cowtowncoder]

@jskierbi It is possible there isn't much change, but 2.9.5 is now out, with 9 fixes to jackson-databind. None of the fixed issues seem to relate to property discovery so probably behavior hasn't changed but I thought it's worth mention.

[jskierbi]

@cowtowncoder it's present starting from 2.9.4 onwards. I've created an issue in jackson-databind: FasterXML/jackson-databind#2001

I've prepared also a fix, PR is here: FasterXML/jackson-databind#2002

[apatrida]

Can we close this issue if it is not a module specific problem @jskierbi ?

[jskierbi]

Yes indeed, this issue is now solved starting from jackson-databind 2.9.6.