How would a shared cache deal with that difference?

It should be fine to share the cache.  It is designed to store information coming from upstream servers, not whatever we generate or send to clients.

You don't even need to split the instances.  It's possible to disable DNS64 on a per-client/query basis.  We have an example with that in our docs: https://www.knot-resolver.cz/documentation/v5.7.3/modules-dns64.html#advanced-options
(sending 5.x link, as you're using that branch apparently)

Around dns64 I'm more worried about other kinds of issues:
https://gitlab.nic.cz/knot/knot-resolver/-/issues/905

--Vladimir