Docs / Optional Verification / Saved Conversation Tests

Test the conversation before changing its maps

An invoice map can produce exactly the expected text and still use the wrong order number. A saved conversation test connects your existing mapping examples, extracts their business facts, and checks their relationships with the same interpreter used for real scenario runs.

1. Reuse the mapping cases you already have

Save a synthetic input and expected output on each bound map using Saved test cases in Mapper. A conversation observation names one of those case IDs, not a second copy of the document. Each referenced map must pass its saved case first. A broken map or mismatched expected output is an execution error for the conversation test, not a successful negative test.

For an incoming map, facts come from the saved input's selected transaction. For an outgoing map, facts come from the map's actual generated X12, never its expected output. Generated documents are validated against the bound syntax tree even when the map's individual case does not request X12 validation. No extra Mapper syntax is needed: binding fact expressions use the existing Mapper evaluator.

2. Add cases to the binding JSON

Edit your binding in the existing Monaco editor and add spec.regressionCases. The generated binding reference and JSON Schema describe every field. There is no separate test repository or authoring language.

The supplier, partial-fulfillment, and carrier bindings each include four reusable cases. Copy the regressionCases array into your own binding after importing the matching map cases, then adapt its step and case IDs. Creating a binding from the form does not automatically copy example cases or placeholder resource IDs.

{
  "id": "accepted-load",
  "name": "Accept, deliver, invoice",
  "parameters": {},
  "observations": [
    { "stepId": "loadTender", "mappingCaseId": "tender-example", "observedAfterSeconds": 0 },
    { "stepId": "tenderResponse", "mappingCaseId": "accept-example", "observedAfterSeconds": 60 },
    { "stepId": "shipmentStatus", "mappingCaseId": "pickup-example", "observedAfterSeconds": 3600 },
    { "stepId": "shipmentStatus", "mappingCaseId": "delivery-example", "observedAfterSeconds": 172800 },
    { "stepId": "freightInvoice", "mappingCaseId": "invoice-example", "observedAfterSeconds": 172860 }
  ],
  "evaluatedAfterSeconds": 172860,
  "closeSteps": ["shipmentStatus"],
  "expected": { "outcome": "PASSED", "checks": [] }
}

Times are whole seconds after a synthetic start at 2000-01-01T00:00:00Z; they do not wait for a clock or alter dates inside the documents. Observation array order assigns occurrence numbers within each step. Receipt-time windows use the supplied offsets; event-date assertions still use extracted facts. Evaluation cannot precede an observation. closeSteps closes only explicitly closable steps at evaluation time, after every listed observation.

Set parameters to the definition's ordinary run parameters. Expect PENDING to test an unfinished conversation, or FAILED to test a specific rejected one. A negative test must name at least one failing check:

"expected": {
  "outcome": "FAILED",
  "checks": [{
    "id": "transition:responseForTender",
    "outcome": "FAILED",
    "code": "TRANSITION_WINDOW_EXPIRED"
  }]
}

A case passes only when the conversation outcome and every named check match. It may therefore show test PASSED · conversation FAILED: the intended rejection occurred. An unrelated failure alone cannot satisfy that expectation. Results show check IDs and codes; they omit raw documents, extracted values, and Mapper diagnostics.

3. Run against the exact proposed configuration

• Browser: apply the binding and its cases through the existing authoring workflow. On the next affected map's Review & deploy, choose Verify saved cases on server. When the configuration contains conversation cases, this action tests the whole proposed configuration, not just that map. Mapping-only workspaces keep the existing simpler check.
• Require the result: the browser's Require this configuration test result when applying option links only a current passing result to Apply. Failed or stale selected evidence blocks Apply; explicitly clear the option if you decide to deploy without that optional gate.
• CI or API: the existing configuration runner verify command and Integration API run both kinds of cases against the desired bundle. Use --verification-run-id to require the server-owned pass during apply. This also tests not-yet-applied binding edits.
• Automatic Git imports: the existing opt-in saved-case policy includes conversation cases. A failed or incomplete test leaves the current configuration unchanged.

Cases remain configuration source in Git. Results remain durable evidence tied to the plan, complete desired bundle, mapping cases, binding and definition hashes, syntax trees, and evaluator. Reopening the same review retrieves the result; applying with its ID records it in Change history. Changed dependencies require fresh verification.

What this does—and does not—prove

Each binding supports 10 cases, 20 observations per case, and 64 KiB of case JSON. A suite supports 25 tested maps, 25 tested bindings, and 100 combined cases within the existing 30-second execution deadline. Coverage explicitly counts maps and bindings with no cases. Cancellation, retention, and admission limits are shared with mapping verification.

Every observed step must use a runtime-mapping target. Adapter-controlled and observation-only steps cannot supply offline observations. This version does not invent reply lineage, MDN/997/999 receipts, or mapping-success evidence for assurance stages. Definitions requiring such evidence will remain pending or fail their deadlines; do not remove a real requirement merely to force a pass. Keep document-only example checks separate from live Test-traffic verification when transport or partner acceptance matters.

The downloadable conversation-cases.json files are ModernEDI's richer development fixtures, not the customer binding schema. The saved cases inside spec.regressionCases are the portable, server-executable customer format.