Counterfactual Scenarios: What Would Flip a Prediction?
Use model.scenario() on a ReasoningModel to find minimal changes that flip a prediction to a target class, returned as structured ScenarioResult objects.
Counterfactual analysis answers a concrete question: “What is the smallest change to this input that would flip the model’s prediction?” Instead of just knowing that a loan application was denied, you can tell the applicant exactly what would need to change (credit score up by 40 points, adding a co-borrower with certain attributes) to receive a different outcome.model.scenario() is available on any ReasoningModel and returns a structured ScenarioResult containing one QueryResult per input row, each with the candidate counterfactual edits found by the search.
By default target_class is 1. Control the search budget with n_trials and max_steps:
result = model.scenario( op.LocalDataset.from_pandas(queries_df).upload(), target_class=1, n_trials=200, # number of attempts per query row max_steps=20, # maximum number of edit steps per attempt)
Increase n_trials when the default search misses plausible counterfactuals for complex, high-dimensional inputs. Increase max_steps when you’re willing to accept counterfactuals that require more simultaneous changes.
import outerproduct as opimport pandas as pdop.init()# Train a modeldataset = op.LocalDataset.from_csv("loans.csv").upload()model = op.reasoning.fit( dataset, task=op.Binclass(label_column="approved")).wait()# Predict on new applicantsX_new = pd.read_csv("new_applicants.csv")test_dataset = op.LocalDataset.from_pandas(X_new).upload()predictions = model.predict(test_dataset)# Run counterfactuals only for denied applicantsimport numpy as npdenied_mask = predictions < 0.5denied_df = X_new[denied_mask].reset_index(drop=True)denied_dataset = op.LocalDataset.from_pandas(denied_df).upload()result = model.scenario(denied_dataset, target_class=1, n_trials=100, max_steps=10)for q in result.queries: print(f"\nApplicant {q.query_index} (baseline={q.baseline_prediction:.2f}):") if q.already_at_desired_class: print(" Already approved, no changes needed.") continue if not q.counterfactuals: print(" No counterfactual found within search budget.") continue # Show the simplest counterfactual (fewest changes) simplest = min(q.counterfactuals, key=lambda cf: cf.n_changes) print(f" Simplest path to approval ({simplest.n_changes} change(s)):") for feature, change in simplest.changes.items(): print(f" {feature}: {change.before} → {change.after}")
