Skip to content

Latest commit

 

History

History
70 lines (60 loc) · 3.74 KB

interactive-command.md

File metadata and controls

70 lines (60 loc) · 3.74 KB

bf orchestrator:interactive

A user can play with an Orchestrator base model interactively and improve a snapshot in real-time.

Introduction

The 'interactive' command starts an interactive session that a user can access an Orchestrator model in real-time doing following:

  • Predict the intent of an input utterance using the 'p' commandlet.
  • Analyze a model example set, by executing the 'v' (validation) commandlet and produce an evaluation report in real-time.
  • Add, remove, or change the intents of an input utterace using the 'a', 'r', and 'c' commandlets, respectively. Users can reference a validation report and choose an ambiguous, misclassified, or low-confidence utterance and change their intent labels.
  • Remove some labels completely from the model example set using the 'rl' commandlet.
  • Create a new model example set snapshot using the 'n' commandlet.

List of interactive commmandlets

Below is a list of the commandlets that can be issued during a 'interactive' session Commandlets: h, q, d, s, u, cu, i, ci, ni, cni, q, p, v, vd, va, vm, vl, vat, vlt, vmt, vut, vo, a, r, c, rl, n

  • h - print this help message
  • q - quit
  • d - display utterance, intent label array inputs, Orchestrator config, and the label-index map
  • s - show label-utterance statistics of the model examples
  • u - enter a new utterance and save it as the "current" utterance input
  • cu - clear the "current" utterance input
  • i - enter an intent and add it to the "current" intent label array input (can be an index for retrieving a label from the label-index map)
  • ci - clear the "current" intent label array input
  • ni - enter an intent and add it to the "new" intent label array input (can be an index for retrieving a label from the label-index map)
  • cni - clear the "new" intent label array input
  • f - find the "current" utterance if it is already in the model example set
  • p - make a prediction on the "current" utterance input
  • v - validate the model and save analyses (validation report) to "experiment_predicting_va\orchestrator_predicting_set_summary.html"
  • vd - reference a validation Duplicates report (previously generated by the "v" command) and enter an index for retrieving utterance/intents into "current"
  • va - reference a validation Ambiguous report (previously generated by the "v" command) and enter an index for retrieving utterance/intents into "current"
  • vm - reference a validation Misclassified report and enter an index (previously generated by the "v" command) for retrieving utterance/intents into "current"
  • vl - reference a validation LowConfidence report (previously generated by the "v" command) and enter an index for retrieving utterance/intents into "current"
  • vat - enter a new validation-report ambiguous closeness threshold
  • vlt - enter a new validation-report low-confidence threshold
  • vmt - enter a new multi-label threshold
  • vut - enter a new unknown-label threshold
  • vo - enter a boolean for obfuscating labels/utterances or not in evaluation reports generated by the "v" command'
  • a - add the "current" utterance and intent labels to the model example set
  • r - remove the "current" utterance and intent labels from the model example set
  • c - remove the "current" utterance's intent labels and then add it with the "new" intent labels to the model example set
  • rl - remove the "current" intent labels from the model example set
  • n - create a new snapshot of model examples and save it to "experiment_predicting_va\orchestrator_predicting_training_set.blu"

Example

$ bf orchestrator:interactive --out=resources\data\Columnar\InteractiveOutput --model=resources\data\Columnar\ModelConfig --in=resources\data\Columnar\Email.blu