8. Conformance Testing and Evaluation (Normative)

8.1 Purpose and Scope

This section defines the normative conformance testing and evaluation requirements for determining whether a system satisfies the ZKS Standard. It operationalizes:

• the ZKS definition and operative semantics (Section 2);
• architectural component constraints and plane separation requirements (Section 3);
• deployment invariants (Section 4);
• threat taxonomy and sovereignty failure conditions (Section 5); and
• conformance requirements and evidence principles (Section 7).

A ZKS conformance determination SHALL be based on capability outcomes rather than documentation assertions. Conformance testing SHALL therefore include negative capability tests (demonstrations that a third party cannot decrypt or revoke decryptability) in addition to architectural mapping and observable-behavior evidence.

This section is normative. A system SHALL NOT claim ZKS compliance unless it is evaluated according to the applicable requirements of this section.

Detailed test procedures and checklists are provided in Appendix D.

8.2 ZKS Conformance Criteria

8.2.1 Conformance Classes

Conformance SHALL be evaluated against the declared compliance profile (Section 2.6):

• ZKS-Core (mandatory baseline)
• ZKS-Enhanced (additional assurance controls)
• ZKS-Enterprise (enterprise governance and evidence requirements)

A claim of ZKS-Enhanced or ZKS-Enterprise implies compliance with all ZKS-Core requirements.

8.2.2 Evaluation Scope Declaration

Before testing begins, the evaluator SHALL produce a scope declaration containing:

1. Deployment model(s) in scope (Section 4).
2. Component inventory mapped to the canonical logical components (CSD, CSS, OP, KCS/UKRS, IRM) (Section 3.2).
3. Plane mapping per canonical diagrams:
   • Data Plane → CSS
   • Control Plane → OP
   • Key Plane → KCS/UKRS (if present)
   • Authorization → OOB (if used)
4. Third-party intermediaries in scope (storage providers, network relays, identity providers, analytics/telemetry services, crash reporting services, managed infrastructure providers, build and distribution infrastructure).
5. Enabled optional features (UKRS/key separation mode, cross-domain collaboration, enterprise delegation, ZTA integration with opaque enforcement points).
6. Declared threat assumptions used for evaluation, at minimum those in Section 5.2.

A conformance evaluation SHALL be considered invalid if the scope declaration omits a component or intermediary that materially participates in ciphertext storage, key custody, orchestration, authorization, recovery, or metadata logging.

8.2.3 Pass/Fail Determination

A system SHALL be deemed non-compliant if any test demonstrates that:

• any third party can decrypt protected user information, induce decryption outside the CSD, or access the complete set of decryption components (given ciphertext availability) (Sections 2.1 and 5.4); or
• any third party can revoke decryptability for an authorized user (Section 2.2.6); or
• any prohibited capability (Section 7.4.4) exists within the evaluated scope.

Partial conformance SHALL NOT be represented as ZKS compliance. Systems that pass some but not all tests MAY be described as 'evaluated against ZKS with identified gaps' but SHALL NOT use 'ZKS-compliant' terminology.

8.3 Test Methodology

8.3.1 Testing Principles

Conformance testing SHALL follow these principles:

1. Capability-based evaluation. Tests MUST validate what components can do, not what they claim to do.

2. Negative capability demonstration. Tests MUST validate the absence of architectural capability for third-party decryption. While verifying absolute impossibility is theoretically limited, the evaluation MUST demonstrate that no defined interface, API, administrative workflow, or storage mechanism exposes plaintext or unwrapped keys, and that standard adversarial techniques fail to extract them.

3. Observable behavior + adversarial simulation. Observable behavior evidence (Section 7.5.2 E1) MUST be complemented with adversarial simulation tests (Sections 8.3.3–8.3.6).

4. Closed-source compatibility. The evaluator SHALL NOT require source code disclosure as a prerequisite for conformance if negative capability and boundary evidence is sufficient to demonstrate cryptographic blindness and incapability, consistent with Section 7.5.

5. Change impact awareness. Evidence and tests SHALL be version-scoped, but SHALL permit inheritance of results when changes are demonstrated to be non-impacting to sovereignty boundaries, key custody semantics, recovery pathways, or plane separation (Section 7.5).

8.3.2 Test Artifacts

A conformance evaluation SHALL produce:

• a test plan mapped to ZKS-Core assertions (A1–A8) and applicable conditional assertions (A9–A12);
• a component and plane mapping report;
• a test execution log;
• evidence artifacts (captures, configuration manifests, results); and
• a conformance determination report with explicit pass/fail rationale.

8.3.3 Baseline Mapping Tests (Structural)

The evaluator SHALL perform structural mapping tests to demonstrate that the implementation is mappable to the reference architecture.

T-STR-1 (Component Mapping). Map all system elements to CSD, CSS, OP, KCS/UKRS, IRM. Demonstrate sovereignty boundary placement. Traceability: Sections 3.2–3.4; Appendix C.

T-STR-2 (Plane Alignment). Demonstrate plane alignment consistent with canonical diagrams, and identify any cross-plane flows. Traceability: Section 3.3; Section 4; Appendix C.

Failure of mapping tests is not automatically non-compliance, but any unmappable component or unexplained cross-plane flow SHALL be treated as a high-risk anomaly requiring targeted testing. Acceptable cross-plane flows are limited to explicit, documented, sovereignty-preserving operations (e.g., delivery notifications or sync coordination signals via OP). If the anomaly cannot be resolved with evidence, the evaluation SHALL result in non-compliance.

8.3.4 Observable Behavior Tests (Black-Box)

T-OBS-1 (Ciphertext-Only Data Plane). Demonstrate that transfers to CSS and between devices do not include plaintext, unwrapped keys, or derivation secrets. Traceability: Sections 3.5.2, 3.6; Section 7.5.2 E1.

T-OBS-2 (OP Key-Plane Blindness). Demonstrate that OP traffic does not contain unwrapping capability and that OP does not log or retain key-plane artifacts in a form that enables derivation of decryptability or correlation. Traceability: Section 3.5.5; Section 4.5; Section 5.3.4.

T-OBS-3 (KCS/UKRS Blindness, if present). Demonstrate that KCS/UKRS stores only wrapped/opaque artifacts and that its interfaces do not expose unwrapped keys or derivation inputs. Traceability: Section 3.5.3; Section 4.10; Section 7.4.3 A9.

8.3.5 Adversarial Capability Tests (Provider/Intermediary Perspective)

These tests validate the core ZKS claim that third parties cannot decrypt or revoke decryptability.

Note: Adversarial capability tests (T-CAP) SHOULD be performed in a dedicated Test or Staging Environment that is cryptographically and configurably identical to the Production Environment. This allows the evaluator to assume privileged roles (e.g., root database access) to verify blindness without violating production data privacy. The test environment MUST use the same key generation and wrapping logic as production, not simplified substitutes. If T-CAP tests are performed in Production, the evaluator SHALL document additional controls to prevent privacy violations.

T-CAP-1 (Provider Infrastructure Access Test). In a Test/Staging environment, grant the evaluator privileges equivalent to the service operator (e.g., shell access to OP/CSS servers, database admin). Demonstrate that even with this access, the evaluator cannot access or derive plaintext user information or unwrapped keys. Traceability: Section 7.5.2 E2; Section 5.3.3.

T-CAP-2 (Collusion Test). Assuming collusion between OP and CSS operators (and KCS/UKRS if present), demonstrate inability to access or derive the complete set of decryption components. Traceability: Tenet 4; Section 5.3.6; Section 7.5.2 E4.

T-CAP-3 (Opaque Enforcement Point Test, if present). If ZTA policy engines or external enforcement points are used, demonstrate that they remain cryptographically blind and do not require decryptability outside CSD. Traceability: Section 6.2.4; Section 7.4.3 A12.

T-CAP-4 (Binary Transparency Verification). Verify that all distributed client binaries are logged to a public, append-only transparency log before distribution. Demonstrate that client software verifies its own binary hash against the transparency log at startup. Attempt to install a binary not present in the log; verify client rejection or fail-secure behavior. Traceability: Section 5.3.1; Section 7.4.4.

8.3.6 Revocation Boundary Tests

T-REV-1 (Availability Failure vs Decryptability). Disable or deny access to CSS and OP services and demonstrate that decryptability remains possible for any ciphertext already available to the user's CSD devices. Traceability: Section 2.2.6; Section 4.6; Section 5.3.2; Section 7.5.2 E3.

T-REV-2 (UKRS Denial Test, if present). Disable or deny UKRS service and demonstrate that decryptability is restorable via the required sovereign restoration pathway (e.g., secondary user-held recovery artifact) given ciphertext availability. The test MUST verify that sovereign restoration does not require any form of provider cooperation, including identity verification. Traceability: Section 4.10 INV-4.10-D; Section 7.4.3 A9; Section 7.5.2 E3.

A failure of T-REV tests SHALL result in non-compliance.

8.4 Evaluation of Optional Features

8.4.1 UKRS / Key Separation Mode

If UKRS is enabled, the evaluator SHALL confirm:

• user initiation and reversibility of key locking;
• cryptographic blindness of UKRS;
• explicit authorization and OOB requirements when applicable; and
• sovereign restoration independent of provider discretionary access.

Traceability: Section 4.10; Section 5.3.2; Section 7.4.3 A9.

8.4.2 Cross-Domain Collaboration

If collaboration is enabled, the evaluator SHALL test for prohibited techniques:

• proxy re-encryption or server-side re-wrapping that provides intermediary access to plaintext or unwrapped keys, even temporarily.

Evidence SHALL include protocol observation and adversarial simulation where feasible.

Traceability: Section 4.8; Section 5.4; Section 7.4.3 A10.

8.4.3 Enterprise Delegation

If enterprise delegation is enabled, the evaluator SHALL validate:

• delegation remains within the user's sovereignty domain; and
• no vendor or external intermediary receives decryptability capability or key possession through governance workflows.

Traceability: Section 3.5.6; Section 4.7; Section 7.4.3 A11.

8.4.4 Split Topology

Where split topology is used, the evaluator SHOULD document:

• distinct operator entities for CSS and OP; and
• reduction of correlated observability through identifier minimization and independent logging.

Split topology is not required for ZKS-Core but strengthens the evidence for Tenet 4 compliance.

ZKS-Enterprise: Systems claiming ZKS-Enterprise SHOULD implement split topology OR document compensating controls with equivalent assurance.

Traceability: Section 4.6; Section 5.3.6.

8.5 Non-Compliance Determination (Normative)

A system SHALL be determined non-compliant if any of the following is observed within the evaluation scope:

1. Decryptability Outside CSD. Any plaintext decryption occurs outside the CSD, or external components can induce decryption.

2. Third-Party Key Possession. Any third party possesses plaintext keys, derivation secrets, or equivalent materials enabling decryptability.

3. Provider Kill Switch. Any third party can revoke decryptability by withholding the only recovery artifact or by controlling essential key release.

4. Prohibited Collaboration Techniques. Any unblinded re-encryption proxy or server-side key wrapping provides intermediary access to plaintext/unwrapped keys.

5. OP/KCS Aggregation Capability. OP or KCS/UKRS acts as a collection point permitting derivation of decryptability through correlation, logging, or retention.

6. Opaque Enforcement Without Evidence. External enforcement points that cannot be demonstrated to be cryptographically blind SHALL be treated as non-compliant. This applies to any component where internal behavior cannot be independently verified, not just policy enforcement points.

7. Evidence Insufficiency for Capability Claims. If required evidence categories (E1–E4) cannot be satisfied for the declared scope, the system SHALL be treated as non-compliant.

8.6 Conformance Reporting (Normative)

The evaluator SHALL produce a conformance report containing:

• declared compliance profile (ZKS-Core / Enhanced / Enterprise);
• scope declaration (Section 8.2.2);
• test plan and mapping to assertions (A1–A12);
• test results with pass/fail and supporting artifacts;
• identified deviations and their impact; and
• a final conformance determination.

Critical Constraint on Deviations: Identified deviations SHALL NOT be accepted for Mandatory Assertions (A1–A8) or Prohibited Capabilities (Section 7.4.4). A deviation in these core areas results in a Non-Compliant determination. Deviations are permissible only for optional features or documentation gaps that do not undermine the ZKS sovereignty definition.

The report SHALL explicitly state any assumptions, limitations, or exclusions, and SHALL not represent partial evaluation as full compliance.

The conformance report SHOULD be countersigned by the evaluated entity acknowledging the accuracy of the scope declaration.