DifferentiationInterface: Fix broken Mooncake version

[wsmoses]

cc @gdalle this will prevent usage of the breaking but not tagged as breaking mooncake release that was causing you issues

[wsmoses]

@DilumAluthge I don't myself have permission to merge, if you can

[gdalle]

Good idea, thanks. Ping @sunxd3

[yebai]

Given the wide range of libraries that depend on both DI and Mooncake, these upper bounds are extremely disruptive. I believe a large collection of libraries in both SciML and Turing currently suffer from this upper bound.

cc @ChrisRackauckas

[ChrisRackauckas]

I will trust that if a package says it needs the upper bounds, then it needs them. Of course I'll be happy when they are lifted but I am sure @gdalle will get to it when he has a moment. Let's all just have a big open source hug because this is all a lot of work

[yebai]

I'd just point out that developing Mooncake is significantly harder than it appears on GitHub -- significant funding and resources are invested out of goodwill to help the Julia community have a robust, performant autograd library.

[DilumAluthge]

I'll defer to the author of the package being modified in this PR, which is DifferentiationInterface.

@gdalle Would you like me to revert this PR?

[gdalle]

Let's please keep it while Mooncake's compatibility with DI is being worked out

[yebai]

I ran the DI tests locally against Mooncake 0.5.27, and the full DI test suite passes without errors.

Could @wsmoses or @gdalle clarify what triggered this PR and provide the specific failures that justify it? As it stands, the change has caused significant disruptions in Turing.jl and SciML.jl.

It would be preferable to report issues upstream to the Mooncake repository before imposing an upper bound in DI. As a general maintenance principle, strict upper bounds should be applied consistently across AD backends. For example, Zygote is frequently broken, and Enzyme is known to segfault in various scenarios. This makes it unclear where a principled boundary should be drawn.

More broadly, decisions of this kind benefit from clear justification. Without that, there is a risk of eroded public trust.

[gdalle]

I ran the DI tests locally against Mooncake 0.5.27, and the full DI test suite passes without errors.

Did you also run the tests with static array scenarios and friendly tangents, which are included in DI's Mooncake test suite? These are the ones that are failing with current versions of Mooncake, see e.g. the CI in JuliaDiff/DifferentiationInterface.jl#994 (specifically this run). It is expected that the standard array tests would pass either way, because Mooncake considers the built-in Array as a self-tangent type.

Could @wsmoses or @gdalle clarify what triggered this PR and provide the specific failures that justify it? As it stands, the change has caused significant disruptions in Turing.jl and SciML.jl.

This PR is justified by the breaking changes Mooncake introduced in chalk-lab/Mooncake.jl#1103. Since then, we've been summarizing the changes in JuliaDiff/DifferentiationInterface.jl#986, and working on a fix in JuliaDiff/DifferentiationInterface.jl#988 with @sunxd3. I discussed this topic on Slack to find a way to support all versions of Mooncake, and I think we came up with something that could work, so it will be implemented soon.

It would be preferable to report issues upstream to the Mooncake repository before imposing an upper bound in DI.

I explicitly pointed out in the Mooncake PR defining the new friendly tangent system that users would suffer from the breakage. No answer was given.
Meanwhile, the disruption started to hit downstream users like @JoshuaLampert in JuliaGeometry/Meshes.jl#1324 (comment) (where I tagged a Mooncake developer)

More broadly, decisions of this kind benefit from clear justification. Without that, there is a risk of eroded public trust.

I'm glad we agree on the risk of eroded public trust in software. My decisions regarding DI are made to safeguard that trust, and sometimes that means protecting users from breaking upstream changes while they are being worked on. That is why, even though this PR was opened by @wsmoses and not me, I ultimately agree with its defensive stance.