DataFrame

An OuterProductDataFrame is a query against the tables in your workspace (defined below). Importantly, a DataFrame is a lazy-query, meaning that it will not materialize data until it is needed for computation. As a result, you can query across data at any scale as if it were in memory. The DataFrame is built in Rust on Apache DataFusion, so that queries run as real relational plans against all registered data sources simultaneously. These can be files on S3, Snowflake, Databricks, and your local machine.

Workspace

The OuterProduct Workspace provides one object to connect and query across multiple data sources. Construct custom views of your data (e.g. filter, join, derive columns) for training models and reasoning over these models.

# Initialize your workspace
ws = op.Workspace()

# Chain operations off a registered table.
train = ws.table("customers").filter(op.col("signup_year") >= 2020)

# Or write SQL against the workspace.
train = ws.sql("SELECT * FROM customers WHERE signup_year >= 2020")

Getting Started

Concepts

Data Connectors

Workspace

​Workspace

Workspace