It turns out that my problem was just an issue with scoping, and was probably made more complex-looking because of the use of refs and groups within the XSD I was given.
To fix the issue raised here, I had to get the <unique> and <keyref> nodes at the same scope. I’m not sure yet if they have to be in the same document or not, but that’s how I did it. There was some minor tweaking needed in the XPath for the unique node, but it was trivial. The result, with both nodes in the root <element> definition:
<xs:unique name="uniqueParameterName"> <xs:selector xpath=".//item/parameter"/> <xs:field xpath="name"/> </xs:unique> <xs:keyref name="engineParameterRef" refer="uniqueParameterName"> <xs:selector xpath="./engine"/> <xs:field xpath="parameter_name"/> </xs:keyref>
And now it validates if there are engine nodes with matching item parameters, and it validates if there are no engine nodes and no item nodes.