Connect your Databricks workspace to your OuterProduct account

Give your OuterProduct account permission to access your Databricks workspace

We authenticate with Databricks via OAuth token federation. OuterProduct gets an AWS IAM OIDC token through STS, then Databricks exchanges it for a Databricks OAuth token using OAuth 2.0 Token Exchange. For Databricks background, see Authenticate access to Databricks using OAuth token federation.

Open the New Connector wizard

Log in to console.outerproduct.com and navigate to Connectors. Click New Connector, give your connector a name, and select Databricks.

Enter your Databricks connection details

Enter your Databricks workspace URL, SQL warehouse HTTP path, and the service principal OuterProduct should authenticate as.

Run the generated federation policy command

Copy the Databricks CLI command from the connector wizard and run it as a Databricks account admin. Databricks federation policies are account-level, so your CLI profile must use https://accounts.cloud.databricks.com. For policy details, see Configure a federation policy.

databricks account service-principal-federation-policy create <service-principal-id> --json '{
  "oidc_policy": {
    "issuer": "https://<outerproduct-aws-issuer>.tokens.sts.global.api.aws",
    "audiences": ["databricks"],
    "subject": "<outerproduct-aws-principal-arn>"
  }
}'

Do not edit the generated issuer, audience, or subject.

Grant Databricks permissions

Grant the Databricks service principal access to the SQL warehouse and tables you want OuterProduct to read. It needs CAN USE on the warehouse plus USE CATALOG, USE SCHEMA, and SELECT on the registered tables.

Save the connector

Return to the connector wizard and click Add connector. OuterProduct validates the workload identity login before saving the connector.

The generated federation policy only sets up login. Databricks table and warehouse permissions still come from the service principal.

Use it from Python

Once your connector is saved in the Console, register the Databricks table as a source table with ws.register_databricks. Pass a table name, the source locator in catalog.schema.table form, and the connector name from the Console. The call returns a DataFrame you can train on, or query later with ws.table(name) or ws.sql(...).

import outerproduct as op

op.init()  # reads OUTERPRODUCT_API_KEY from the environment
ws = op.Workspace()

ws.register_databricks("customers", "main.default.customers", connector="prod-databricks")

# Build the training set as a query over the registered source.
train = ws.table("customers").filter(op.col("signup_year") >= 2020)

model = op.reasoning.fit(train, task=op.Binclass(label_column="churn")).wait()

Getting Started

Concepts

Data Connectors

Connect your Databricks workspace to your OuterProduct account

Give your OuterProduct account permission to access your Databricks workspace

Use it from Python

​Give your OuterProduct account permission to access your Databricks workspace

​Use it from Python

Give your OuterProduct account permission to access your Databricks workspace

Use it from Python