Registration

Providers

Azure Blob Store

Register an Azure Blob Store provider.

Azure Blob Storage can be used as the storage component for Spark or the Featureform Pandas Runner.

Examples:

blob = ff.register_blob_store(
    name="azure-quickstart",
    container_name="my_company_container"
    root_path="custom/path/in/container"
    account_name=<azure_account_name>
    account_key=<azure_account_key>
    description="An azure blob store provider to store offline and inference data"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Azure blob store to be registered	required
container_name	str	(Immutable) Azure container name	required
root_path	str	(Immutable) A custom path in container to store data	required
account_name	str	(Immutable) Azure account name	required
account_key	str	(Mutable) Secret azure account key	required
description	str	(Mutable) Description of Azure Blob provider to be registered	''
team	str	(Mutable) The name of the team registering the filestore	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	None
properties	dict	(Mutable) Optional grouping mechanism for resources	None

Returns:

Name	Type	Description
blob	StorageProvider	Provider has all the functionality of OnlineProvider

BigQuery

Register a BigQuery provider.

Examples:

bigquery = ff.register_bigquery(
    name="bigquery-quickstart",
    description="A BigQuery deployment we created for the Featureform quickstart",
    project_id="quickstart-project",
    dataset_id="quickstart-dataset",
    credentials=GCPCredentials(...)
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of BigQuery provider to be registered	required
project_id	str	(Immutable) The Project name in GCP	required
dataset_id	str	(Immutable) The Dataset name in GCP under the Project Id	required
credentials	GCPCredentials	(Mutable) GCP credentials to access BigQuery	required
description	str	(Mutable) Description of BigQuery provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
bigquery	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_bigquery(
    self,
    name: str,
    project_id: str,
    dataset_id: str,
    credentials: GCPCredentials,
    credentials_path: str = "",
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a BigQuery provider.

    **Examples**:
    ```
    bigquery = ff.register_bigquery(
        name="bigquery-quickstart",
        description="A BigQuery deployment we created for the Featureform quickstart",
        project_id="quickstart-project",
        dataset_id="quickstart-dataset",
        credentials=GCPCredentials(...)
    )
    ```

    Args:
        name (str): (Immutable) Name of BigQuery provider to be registered
        project_id (str): (Immutable) The Project name in GCP
        dataset_id (str): (Immutable) The Dataset name in GCP under the Project Id
        credentials (GCPCredentials): (Mutable) GCP credentials to access BigQuery
        description (str): (Mutable) Description of BigQuery provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        bigquery (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)

    config = BigQueryConfig(
        project_id=project_id,
        dataset_id=dataset_id,
        credentials=credentials,
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OfflineSQLProvider(self, provider)

Cassandra

Register a Cassandra provider.

Examples:

cassandra = ff.register_cassandra(
        name = "cassandra",
        description = "Example inference store",
        team = "Featureform",
        host = "0.0.0.0",
        port = 9042,
        username = "cassandra",
        password = "cassandra",
        consistency = "THREE",
        replication = 3
    )

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Cassandra provider to be registered	required
host	str	(Immutable) DNS name of Cassandra	required
port	str	(Mutable) Port	required
username	str	(Mutable) Username	required
password	str	(Mutable) Password	required
consistency	str	(Mutable) Consistency	'THREE'
replication	int	(Mutable) Replication	3
description	str	(Mutable) Description of Cassandra provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
cassandra	OnlineProvider	Provider

Source code in src/featureform/register.py

def register_cassandra(
    self,
    name: str,
    host: str,
    port: int,
    username: str,
    password: str,
    keyspace: str,
    consistency: str = "THREE",
    replication: int = 3,
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Cassandra provider.

    **Examples**:
    ```
    cassandra = ff.register_cassandra(
            name = "cassandra",
            description = "Example inference store",
            team = "Featureform",
            host = "0.0.0.0",
            port = 9042,
            username = "cassandra",
            password = "cassandra",
            consistency = "THREE",
            replication = 3
        )
    ```

    Args:
        name (str): (Immutable) Name of Cassandra provider to be registered
        host (str): (Immutable) DNS name of Cassandra
        port (str): (Mutable) Port
        username (str): (Mutable) Username
        password (str): (Mutable) Password
        consistency (str): (Mutable) Consistency
        replication (int): (Mutable) Replication
        description (str): (Mutable) Description of Cassandra provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        cassandra (OnlineProvider): Provider
    """
    config = CassandraConfig(
        host=host,
        port=port,
        username=username,
        password=password,
        keyspace=keyspace,
        consistency=consistency,
        replication=replication,
    )
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

DynamoDB

Register a DynamoDB provider.

Examples:

dynamodb = ff.register_dynamodb(
    name="dynamodb-quickstart",
    description="A Dynamodb deployment we created for the Featureform quickstart",
    credentials=aws_creds,
    region="us-east-1"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of DynamoDB provider to be registered	required
region	str	(Immutable) Region to create dynamo tables	required
credentials	AWSCredentials	(Mutable) AWS credentials with permissions to create DynamoDB tables	required
should_import_from_s3	bool	(Mutable) Determines whether feature materialization will occur via a direct import of data from S3 to new table (see docs for details)	False
description	str	(Mutable) Description of DynamoDB provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
dynamodb	OnlineProvider	Provider

Source code in src/featureform/register.py

def register_dynamodb(
    self,
    name: str,
    credentials: AWSCredentials,
    region: str,
    should_import_from_s3: bool = False,
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a DynamoDB provider.

    **Examples**:
    ```
    dynamodb = ff.register_dynamodb(
        name="dynamodb-quickstart",
        description="A Dynamodb deployment we created for the Featureform quickstart",
        credentials=aws_creds,
        region="us-east-1"
    )
    ```

    Args:
        name (str): (Immutable) Name of DynamoDB provider to be registered
        region (str): (Immutable) Region to create dynamo tables
        credentials (AWSCredentials): (Mutable) AWS credentials with permissions to create DynamoDB tables
        should_import_from_s3 (bool): (Mutable) Determines whether feature materialization will occur via a direct import of data from S3 to new table (see [docs](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.HowItWorks.html) for details)
        description (str): (Mutable) Description of DynamoDB provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        dynamodb (OnlineProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = DynamodbConfig(
        access_key=credentials.access_key,
        secret_key=credentials.secret_key,
        region=region,
        should_import_from_s3=should_import_from_s3,
    )
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

Firestore

Register a Firestore provider.

Examples:

firestore = ff.register_firestore(
    name="firestore-quickstart",
    description="A Firestore deployment we created for the Featureform quickstart",
    project_id="quickstart-project",
    collection="quickstart-collection",
    credentials=ff.GCPCredentials(...)
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Firestore provider to be registered	required
project_id	str	(Immutable) The Project name in GCP	required
collection	str	(Immutable) The Collection name in Firestore under the given project ID	required
credentials	GCPCredentials	(Mutable) GCP credentials to access Firestore	required
description	str	(Mutable) Description of Firestore provider to be registered	''
team	str	(Mutable) The name of the team registering the filestore	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
firestore	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_firestore(
    self,
    name: str,
    collection: str,
    project_id: str,
    credentials: GCPCredentials,
    credentials_path: str = "",
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Firestore provider.

    **Examples**:
    ```
    firestore = ff.register_firestore(
        name="firestore-quickstart",
        description="A Firestore deployment we created for the Featureform quickstart",
        project_id="quickstart-project",
        collection="quickstart-collection",
        credentials=ff.GCPCredentials(...)
    )
    ```

    Args:
        name (str): (Immutable) Name of Firestore provider to be registered
        project_id (str): (Immutable) The Project name in GCP
        collection (str): (Immutable) The Collection name in Firestore under the given project ID
        credentials (GCPCredentials): (Mutable) GCP credentials to access Firestore
        description (str): (Mutable) Description of Firestore provider to be registered
        team (str): (Mutable) The name of the team registering the filestore
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        firestore (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = FirestoreConfig(
        collection=collection,
        project_id=project_id,
        credentials=credentials,
    )
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

Google Cloud Storage

Register a GCS store provider.

Examples:

gcs = ff.register_gcs(
    name="gcs-quickstart",
    credentials=ff.GCPCredentials(...),
    bucket_name="bucket_name",
    root_path="featureform/path/",
    description="An gcs store provider to store offline"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of GCS store to be registered	required
bucket_name	str	(Immutable) The bucket name	required
root_path	str	(Immutable) Custom path to be used by featureform	required
credentials	GCPCredentials	(Mutable) GCP credentials to access the bucket	required
description	str	(Mutable) Description of GCS provider to be registered	''
team	str	(Mutable) The name of the team registering the filestore	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
gcs	FileStoreProvider	Provider has all the functionality of OfflineProvider

Source code in src/featureform/register.py

def register_gcs(
    self,
    name: str,
    bucket_name: str,
    root_path: str,
    credentials: GCPCredentials,
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a GCS store provider.

    **Examples**:
    ```
    gcs = ff.register_gcs(
        name="gcs-quickstart",
        credentials=ff.GCPCredentials(...),
        bucket_name="bucket_name",
        root_path="featureform/path/",
        description="An gcs store provider to store offline"
    )
    ```

    Args:
        name (str): (Immutable) Name of GCS store to be registered
        bucket_name (str): (Immutable) The bucket name
        root_path (str): (Immutable) Custom path to be used by featureform
        credentials (GCPCredentials): (Mutable) GCP credentials to access the bucket
        description (str): (Mutable) Description of GCS provider to be registered
        team (str): (Mutable) The name of the team registering the filestore
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        gcs (FileStoreProvider): Provider
            has all the functionality of OfflineProvider
    """
    tags, properties = set_tags_properties(tags, properties)

    if bucket_name == "":
        raise ValueError("bucket_name is required and cannot be empty string")

    bucket_name = bucket_name.replace("gs://", "")
    if "/" in bucket_name:
        raise ValueError(
            "bucket_name cannot contain '/'. bucket_name should be the name of the GCS bucket only."
        )

    gcs_config = GCSFileStoreConfig(
        bucket_name=bucket_name, bucket_path=root_path, credentials=credentials
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=gcs_config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return FileStoreProvider(self, provider, gcs_config, gcs_config.type())

HDFS

Register a HDFS store provider.

This has the functionality of an offline store and can be used as a parameter to a k8s or spark provider

Examples:

hdfs = ff.register_hdfs(
    name="hdfs-quickstart",
    host="<host>",
    port="<port>",
    path="<path>",
    username="<username>",
    description="An hdfs store provider to store offline"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of HDFS store to be registered	required
host	str	(Immutable) The hostname for HDFS	required
path	str	(Immutable) A storage path within HDFS	''
port	str	(Mutable) The IPC port for the Namenode for HDFS. (Typically 8020 or 9000)	required
username	str	(Mutable) A Username for HDFS	''
description	str	(Mutable) Description of HDFS provider to be registered	''
team	str	(Mutable) The name of the team registering HDFS	''

Returns:

Name	Type	Description
hdfs	FileStoreProvider	Provider

Source code in src/featureform/register.py

def register_hdfs(
    self,
    name: str,
    host: str,
    port: str,
    username: str = "",
    path: str = "",
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a HDFS store provider.

    This has the functionality of an offline store and can be used as a parameter
    to a k8s or spark provider

    **Examples**:
    ```
    hdfs = ff.register_hdfs(
        name="hdfs-quickstart",
        host="<host>",
        port="<port>",
        path="<path>",
        username="<username>",
        description="An hdfs store provider to store offline"
    )
    ```

    Args:
        name (str): (Immutable) Name of HDFS store to be registered
        host (str): (Immutable) The hostname for HDFS
        path (str): (Immutable) A storage path within HDFS
        port (str): (Mutable) The IPC port for the Namenode for HDFS. (Typically 8020 or 9000)
        username (str): (Mutable) A Username for HDFS
        description (str): (Mutable) Description of HDFS provider to be registered
        team (str): (Mutable) The name of the team registering HDFS

    Returns:
        hdfs (FileStoreProvider): Provider
    """

    hdfs_config = HDFSConfig(host=host, port=port, path=path, username=username)

    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=hdfs_config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return FileStoreProvider(self, provider, hdfs_config, hdfs_config.type())

MongoDB

Register a MongoDB provider.

Examples:

mongodb = ff.register_mongodb(
    name="mongodb-quickstart",
    description="A MongoDB deployment",
    username="my_username",
    password="myPassword",
    database="featureform_database"
    host="my-mongodb.host.com",
    port="10225",
    throughput=10000
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of MongoDB provider to be registered	required
database	str	(Immutable) MongoDB database	required
host	str	(Immutable) MongoDB hostname	required
port	str	(Immutable) MongoDB port	required
username	str	(Mutable) MongoDB username	required
password	str	(Mutable) MongoDB password	required
throughput	int	(Mutable) The maximum RU limit for autoscaling in CosmosDB	1000
description	str	(Mutable) Description of MongoDB provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
mongodb	OnlineProvider	Provider

Source code in src/featureform/register.py

def register_mongodb(
    self,
    name: str,
    username: str,
    password: str,
    database: str,
    host: str,
    port: str,
    throughput: int = 1000,
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a MongoDB provider.

    **Examples**:
    ```
    mongodb = ff.register_mongodb(
        name="mongodb-quickstart",
        description="A MongoDB deployment",
        username="my_username",
        password="myPassword",
        database="featureform_database"
        host="my-mongodb.host.com",
        port="10225",
        throughput=10000
    )
    ```

    Args:
        name (str): (Immutable) Name of MongoDB provider to be registered
        database (str): (Immutable) MongoDB database
        host (str): (Immutable) MongoDB hostname
        port (str): (Immutable) MongoDB port
        username (str): (Mutable) MongoDB username
        password (str): (Mutable) MongoDB password
        throughput (int): (Mutable) The maximum RU limit for autoscaling in CosmosDB
        description (str): (Mutable) Description of MongoDB provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        mongodb (OnlineProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = MongoDBConfig(
        username=username,
        password=password,
        host=host,
        port=port,
        database=database,
        throughput=throughput,
    )
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

Pinecone

Register a Pinecone provider.

Examples:

pinecone = ff.register_pinecone(
    name="pinecone-quickstart",
    project_id="2g13ek7",
    environment="us-west4-gcp-free",
    api_key="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Pinecone provider to be registered	required
project_id	str	(Immutable) Pinecone project id	required
environment	str	(Immutable) Pinecone environment	required
api_key	str	(Mutable) Pinecone api key	required
description	str	(Mutable) Description of Pinecone provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
pinecone	OnlineProvider	Provider

Source code in src/featureform/register.py

def register_pinecone(
    self,
    name: str,
    project_id: str,
    environment: str,
    api_key: str,
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Pinecone provider.

    **Examples**:
    ```
    pinecone = ff.register_pinecone(
        name="pinecone-quickstart",
        project_id="2g13ek7",
        environment="us-west4-gcp-free",
        api_key="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    )
    ```

    Args:
        name (str): (Immutable) Name of Pinecone provider to be registered
        project_id (str): (Immutable) Pinecone project id
        environment (str): (Immutable) Pinecone environment
        api_key (str): (Mutable) Pinecone api key
        description (str): (Mutable) Description of Pinecone provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        pinecone (OnlineProvider): Provider
    """

    tags, properties = set_tags_properties(tags, properties)
    config = PineconeConfig(
        project_id=project_id, environment=environment, api_key=api_key
    )
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

Postgres

Register a Postgres provider.

Examples:

postgres = ff.register_postgres(
    name="postgres-quickstart",
    description="A Postgres deployment we created for the Featureform quickstart",
    host="quickstart-postgres",  # The internal dns name for postgres
    port="5432",
    user="postgres",
    password="password", #pragma: allowlist secret
    database="postgres"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Postgres provider to be registered	required
host	str	(Immutable) Hostname for Postgres	required
database	str	(Immutable) Database	required
port	str	(Mutable) Port	'5432'
user	str	(Mutable) User	required
password	str	(Mutable) Password	required
sslmode	str	(Mutable) SSL mode	'disable'
description	str	(Mutable) Description of Postgres provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
postgres	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_postgres(
    self,
    name: str,
    host: str,
    user: str,
    password: str,
    database: str,
    port: str = "5432",
    description: str = "",
    team: str = "",
    sslmode: str = "disable",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Postgres provider.

    **Examples**:
    ```
    postgres = ff.register_postgres(
        name="postgres-quickstart",
        description="A Postgres deployment we created for the Featureform quickstart",
        host="quickstart-postgres",  # The internal dns name for postgres
        port="5432",
        user="postgres",
        password="password", #pragma: allowlist secret
        database="postgres"
    )
    ```

    Args:
        name (str): (Immutable) Name of Postgres provider to be registered
        host (str): (Immutable) Hostname for Postgres
        database (str): (Immutable) Database
        port (str): (Mutable) Port
        user (str): (Mutable) User
        password (str): (Mutable) Password
        sslmode (str): (Mutable) SSL mode
        description (str): (Mutable) Description of Postgres provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        postgres (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = PostgresConfig(
        host=host,
        port=port,
        database=database,
        user=user,
        password=password,
        sslmode=sslmode,
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=config,
        tags=tags or [],
        properties=properties or {},
    )

    self.__resources.append(provider)
    return OfflineSQLProvider(self, provider)

Redis

Register a Redis provider.

Examples:

redis = ff.register_redis(
    name="redis-quickstart",
    host="quickstart-redis",
    port=6379,
    password="password",
    description="A Redis deployment we created for the Featureform quickstart"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Redis provider to be registered	required
host	str	(Immutable) Hostname for Redis	required
db	str	(Immutable) Redis database number	0
port	int	(Mutable) Redis port	6379
password	str	(Mutable) Redis password	''
description	str	(Mutable) Description of Redis provider to be registered	''
team	str	(Mutable) Name of team	''
tags	Optional[List[str]]	(Mutable) Optional grouping mechanism for resources	None
properties	Optional[dict]	(Mutable) Optional grouping mechanism for resources	None

Returns:

Name	Type	Description
redis	OnlineProvider	Provider

Source code in src/featureform/register.py

def register_redis(
    self,
    name: str,
    host: str,
    port: int = 6379,
    db: int = 0,
    password: str = "",
    description: str = "",
    team: str = "",
    tags: Optional[List[str]] = None,
    properties: Optional[dict] = None,
):
    """Register a Redis provider.

    **Examples**:
    ```
    redis = ff.register_redis(
        name="redis-quickstart",
        host="quickstart-redis",
        port=6379,
        password="password",
        description="A Redis deployment we created for the Featureform quickstart"
    )
    ```

    Args:
        name (str): (Immutable) Name of Redis provider to be registered
        host (str): (Immutable) Hostname for Redis
        db (str): (Immutable) Redis database number
        port (int): (Mutable) Redis port
        password (str): (Mutable) Redis password
        description (str): (Mutable) Description of Redis provider to be registered
        team (str): (Mutable) Name of team
        tags (Optional[List[str]]): (Mutable) Optional grouping mechanism for resources
        properties (Optional[dict]): (Mutable) Optional grouping mechanism for resources

    Returns:
        redis (OnlineProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = RedisConfig(host=host, port=port, password=password, db=db)
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

Redshift

Register a Redshift provider.

Examples:

redshift = ff.register_redshift(
    name="redshift-quickstart",
    description="A Redshift deployment we created for the Featureform quickstart",
    host="quickstart-redshift",  # The internal dns name for redshift
    port="5432",
    user="redshift",
    password="password", #pragma: allowlist secret
    database="dev"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Redshift provider to be registered	required
host	str	(Immutable) Hostname for Redshift	required
database	str	(Immutable) Redshift database	required
port	str	(Mutable) Port	required
user	str	(Mutable) User	required
password	str	(Mutable) Redshift password	required
sslmode	str	(Mutable) SSL mode	'disable'
description	str	(Mutable) Description of Redshift provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
redshift	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_redshift(
    self,
    name: str,
    host: str,
    port: str,
    user: str,
    password: str,
    database: str,
    description: str = "",
    team: str = "",
    sslmode: str = "disable",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Redshift provider.

    **Examples**:
    ```
    redshift = ff.register_redshift(
        name="redshift-quickstart",
        description="A Redshift deployment we created for the Featureform quickstart",
        host="quickstart-redshift",  # The internal dns name for redshift
        port="5432",
        user="redshift",
        password="password", #pragma: allowlist secret
        database="dev"
    )
    ```

    Args:
        name (str): (Immutable) Name of Redshift provider to be registered
        host (str): (Immutable) Hostname for Redshift
        database (str): (Immutable) Redshift database
        port (str): (Mutable) Port
        user (str): (Mutable) User
        password (str): (Mutable) Redshift password
        sslmode (str): (Mutable) SSL mode
        description (str): (Mutable) Description of Redshift provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        redshift (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = RedshiftConfig(
        host=host,
        port=port,
        database=database,
        user=user,
        password=password,
        sslmode=sslmode,
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OfflineSQLProvider(self, provider)

ClickHouse

Register a ClickHouse provider.

Examples:

clickhouse = ff.register_clickhouse(
    name="clickhouse-quickstart",
    description="A ClickHouse deployment we created for the Featureform quickstart",
    host="quickstart-clickhouse",  # The internal dns name for clickhouse
    port=9000,
    user="default",
    password="", #pragma: allowlist secret
    database="default"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of ClickHouse provider to be registered	required
host	str	(Immutable) Hostname for ClickHouse	required
database	str	(Immutable) ClickHouse database	required
port	int	(Mutable) Port	9000
ssl	bool	(Mutable) Enable SSL	False
user	str	(Mutable) User	required
password	str	(Mutable) ClickHouse password	required
description	str	(Mutable) Description of ClickHouse provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
clickhouse	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_clickhouse(
    self,
    name: str,
    host: str,
    user: str,
    password: str,
    database: str,
    port: int = 9000,
    description: str = "",
    team: str = "",
    ssl: bool = False,
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a ClickHouse provider.

    **Examples**:
    ```
    clickhouse = ff.register_clickhouse(
        name="clickhouse-quickstart",
        description="A ClickHouse deployment we created for the Featureform quickstart",
        host="quickstart-clickhouse",  # The internal dns name for clickhouse
        port=9000,
        user="default",
        password="", #pragma: allowlist secret
        database="default"
    )
    ```

    Args:
        name (str): (Immutable) Name of ClickHouse provider to be registered
        host (str): (Immutable) Hostname for ClickHouse
        database (str): (Immutable) ClickHouse database
        port (int): (Mutable) Port
        ssl (bool): (Mutable) Enable SSL
        user (str): (Mutable) User
        password (str): (Mutable) ClickHouse password
        description (str): (Mutable) Description of ClickHouse provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        clickhouse (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = ClickHouseConfig(
        host=host,
        port=port,
        database=database,
        user=user,
        password=password,
        ssl=ssl,
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OfflineSQLProvider(self, provider)

S3

Register a S3 store provider.

This has the functionality of an offline store and can be used as a parameter to a k8s or spark provider

Examples:

s3 = ff.register_s3(
    name="s3-quickstart",
    credentials=aws_creds,
    bucket_name="bucket_name",
    bucket_region=<bucket_region>,
    path="path/to/store/featureform_files/in/",
    description="An s3 store provider to store offline"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of S3 store to be registered	required
bucket_name	str	(Immutable) AWS Bucket Name	required
bucket_region	str	(Immutable) AWS region the bucket is located in	required
path	str	(Immutable) The path used to store featureform files in	''
credentials	AWSCredentials	(Mutable) AWS credentials to access the bucket	required
description	str	(Mutable) Description of S3 provider to be registered	''
team	str	(Mutable) The name of the team registering the filestore	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
s3	FileStoreProvider	Provider has all the functionality of OfflineProvider

Source code in src/featureform/register.py

def register_s3(
    self,
    name: str,
    credentials: AWSCredentials,
    bucket_region: str,
    bucket_name: str,
    path: str = "",
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a S3 store provider.

    This has the functionality of an offline store and can be used as a parameter
    to a k8s or spark provider

    **Examples**:
    ```
    s3 = ff.register_s3(
        name="s3-quickstart",
        credentials=aws_creds,
        bucket_name="bucket_name",
        bucket_region=<bucket_region>,
        path="path/to/store/featureform_files/in/",
        description="An s3 store provider to store offline"
    )
    ```

    Args:
        name (str): (Immutable) Name of S3 store to be registered
        bucket_name (str): (Immutable) AWS Bucket Name
        bucket_region (str): (Immutable) AWS region the bucket is located in
        path (str): (Immutable) The path used to store featureform files in
        credentials (AWSCredentials): (Mutable) AWS credentials to access the bucket
        description (str): (Mutable) Description of S3 provider to be registered
        team (str): (Mutable) The name of the team registering the filestore
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        s3 (FileStoreProvider): Provider
            has all the functionality of OfflineProvider
    """
    tags, properties = set_tags_properties(tags, properties)

    if bucket_name == "":
        raise ValueError("bucket_name is required and cannot be empty string")

    # TODO: add verification into S3StoreConfig
    bucket_name = bucket_name.replace("s3://", "").replace("s3a://", "")

    if "/" in bucket_name:
        raise ValueError(
            "bucket_name cannot contain '/'. bucket_name should be the name of the AWS S3 bucket only."
        )

    s3_config = S3StoreConfig(
        bucket_path=bucket_name,
        bucket_region=bucket_region,
        credentials=credentials,
        path=path,
    )

    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=s3_config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return FileStoreProvider(self, provider, s3_config, s3_config.type())

Snowflake

Current

Register a Snowflake provider.

Examples:

snowflake = ff.register_snowflake(
    name="snowflake-quickstart",
    username="snowflake",
    password="password", #pragma: allowlist secret
    account="account",
    organization="organization",
    database="snowflake",
    schema="PUBLIC",
    description="A Snowflake deployment we created for the Featureform quickstart"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Snowflake provider to be registered	required
account	str	(Immutable) Account	required
organization	str	(Immutable) Organization	required
database	str	(Immutable) Database	required
schema	str	(Immutable) Schema	'PUBLIC'
username	str	(Mutable) Username	required
password	str	(Mutable) Password	required
warehouse	str	(Mutable) Specifies the virtual warehouse to use by default for queries, loading, etc.	''
role	str	(Mutable) Specifies the role to use by default for accessing Snowflake objects in the client session	''
description	str	(Mutable) Description of Snowflake provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
snowflake	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_snowflake(
    self,
    name: str,
    username: str,
    password: str,
    account: str,
    organization: str,
    database: str,
    schema: str = "PUBLIC",
    description: str = "",
    team: str = "",
    warehouse: str = "",
    role: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Snowflake provider.

    **Examples**:
    ```
    snowflake = ff.register_snowflake(
        name="snowflake-quickstart",
        username="snowflake",
        password="password", #pragma: allowlist secret
        account="account",
        organization="organization",
        database="snowflake",
        schema="PUBLIC",
        description="A Snowflake deployment we created for the Featureform quickstart"
    )
    ```

    Args:
        name (str): (Immutable) Name of Snowflake provider to be registered
        account (str): (Immutable) Account
        organization (str): (Immutable) Organization
        database (str): (Immutable) Database
        schema (str): (Immutable) Schema
        username (str): (Mutable) Username
        password (str): (Mutable) Password
        warehouse (str): (Mutable) Specifies the virtual warehouse to use by default for queries, loading, etc.
        role (str): (Mutable) Specifies the role to use by default for accessing Snowflake objects in the client session
        description (str): (Mutable) Description of Snowflake provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        snowflake (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = SnowflakeConfig(
        account=account,
        database=database,
        organization=organization,
        username=username,
        password=password,
        schema=schema,
        warehouse=warehouse,
        role=role,
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OfflineSQLProvider(self, provider)

Legacy

Register a Snowflake provider using legacy credentials.

Examples:

snowflake = ff.register_snowflake_legacy(
    name="snowflake-quickstart",
    username="snowflake",
    password="password",
    account_locator="account-locator",
    database="snowflake",
    schema="PUBLIC",
    description="A Snowflake deployment we created for the Featureform quickstart"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Snowflake provider to be registered	required
account_locator	str	(Immutable) Account Locator	required
schema	str	(Immutable) Schema	'PUBLIC'
database	str	(Immutable) Database	required
username	str	(Mutable) Username	required
password	str	(Mutable) Password	required
warehouse	str	(Mutable) Specifies the virtual warehouse to use by default for queries, loading, etc.	''
role	str	(Mutable) Specifies the role to use by default for accessing Snowflake objects in the client session	''
description	str	(Mutable) Description of Snowflake provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
snowflake	OfflineSQLProvider	Provider

Source code in src/featureform/register.py

def register_snowflake_legacy(
    self,
    name: str,
    username: str,
    password: str,
    account_locator: str,
    database: str,
    schema: str = "PUBLIC",
    description: str = "",
    team: str = "",
    warehouse: str = "",
    role: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Snowflake provider using legacy credentials.

    **Examples**:
    ```
    snowflake = ff.register_snowflake_legacy(
        name="snowflake-quickstart",
        username="snowflake",
        password="password",
        account_locator="account-locator",
        database="snowflake",
        schema="PUBLIC",
        description="A Snowflake deployment we created for the Featureform quickstart"
    )
    ```

    Args:
        name (str): (Immutable) Name of Snowflake provider to be registered
        account_locator (str): (Immutable) Account Locator
        schema (str): (Immutable) Schema
        database (str): (Immutable) Database
        username (str): (Mutable) Username
        password (str): (Mutable) Password
        warehouse (str): (Mutable) Specifies the virtual warehouse to use by default for queries, loading, etc.
        role (str): (Mutable) Specifies the role to use by default for accessing Snowflake objects in the client session
        description (str): (Mutable) Description of Snowflake provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        snowflake (OfflineSQLProvider): Provider
    """
    tags, properties = set_tags_properties(tags, properties)
    config = SnowflakeConfig(
        account_locator=account_locator,
        database=database,
        username=username,
        password=password,
        schema=schema,
        warehouse=warehouse,
        role=role,
    )
    provider = Provider(
        name=name,
        function="OFFLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OfflineSQLProvider(self, provider)

Weaviate

Register a Weaviate provider.

Examples:

weaviate = ff.register_weaviate(
    name="weaviate-quickstart",
    url="https://<CLUSTER NAME>.weaviate.network",
    api_key="<API KEY>"
    description="A Weaviate project for using embeddings in Featureform"
)

Parameters:

Name	Type	Description	Default
name	str	(Immutable) Name of Weaviate provider to be registered	required
url	str	(Immutable) Endpoint of Weaviate cluster, either in the cloud or via another deployment operation	required
api_key	str	(Mutable) Weaviate api key	required
description	str	(Mutable) Description of Weaviate provider to be registered	''
team	str	(Mutable) Name of team	''
tags	List[str]	(Mutable) Optional grouping mechanism for resources	[]
properties	dict	(Mutable) Optional grouping mechanism for resources	{}

Returns:

Name	Type	Description
weaviate	OnlineProvider	Provider

Source code in src/featureform/register.py

def register_weaviate(
    self,
    name: str,
    url: str,
    api_key: str,
    description: str = "",
    team: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a Weaviate provider.

    **Examples**:
    ```
    weaviate = ff.register_weaviate(
        name="weaviate-quickstart",
        url="https://<CLUSTER NAME>.weaviate.network",
        api_key="<API KEY>"
        description="A Weaviate project for using embeddings in Featureform"
    )
    ```

    Args:
        name (str): (Immutable) Name of Weaviate provider to be registered
        url (str): (Immutable) Endpoint of Weaviate cluster, either in the cloud or via another deployment operation
        api_key (str): (Mutable) Weaviate api key
        description (str): (Mutable) Description of Weaviate provider to be registered
        team (str): (Mutable) Name of team
        tags (List[str]): (Mutable) Optional grouping mechanism for resources
        properties (dict): (Mutable) Optional grouping mechanism for resources

    Returns:
        weaviate (OnlineProvider): Provider
    """
    config = WeaviateConfig(url=url, api_key=api_key)
    provider = Provider(
        name=name,
        function="ONLINE",
        description=description,
        team=team,
        config=config,
        tags=tags,
        properties=properties,
    )
    self.__resources.append(provider)
    return OnlineProvider(self, provider)

Primary Sources

Register a SQL table as a primary data source.

Example

postgres = client.get_provider("my_postgres")
table =  postgres.register_table(
    name="transactions",
    variant="july_2023",
    table="transactions_table",
):

Parameters:

Name	Type	Description	Default
name	str	Name of table to be registered	required
variant	str	Name of variant to be registered	''
table	str	Name of SQL table	required
owner	Union[str, UserRegistrar]	Owner	''
description	str	Description of table to be registered	''

Returns:

Name	Type	Description
source	ColumnSourceRegistrar	source

Source code in src/featureform/register.py

def register_table(
    self,
    name: str,
    table: str,
    variant: str = "",
    owner: Union[str, UserRegistrar] = "",
    description: str = "",
    tags: List[str] = [],
    properties: dict = {},
):
    """Register a SQL table as a primary data source.

    **Example**

    ```
    postgres = client.get_provider("my_postgres")
    table =  postgres.register_table(
        name="transactions",
        variant="july_2023",
        table="transactions_table",
    ):
    ```

    Args:
        name (str): Name of table to be registered
        variant (str): Name of variant to be registered
        table (str): Name of SQL table
        owner (Union[str, UserRegistrar]): Owner
        description (str): Description of table to be registered

    Returns:
        source (ColumnSourceRegistrar): source
    """
    return self.__registrar.register_primary_data(
        name=name,
        variant=variant,
        location=SQLTable(table),
        owner=owner,
        provider=self.name(),
        description=description,
        tags=tags,
        properties=properties,
    )

Transformations

featureform.register.ResourceClient

The resource client is used to retrieve information on specific resources (entities, providers, features, labels, training sets, models, users).

Parameters:

Name	Type	Description	Default
host	str	The hostname of the Featureform instance.	None
insecure	bool	True if connecting to an insecure Featureform endpoint. False if using a self-signed or public TLS certificate	False
cert_path	str	The path to a public certificate if using a self-signed certificate.	None

Using the Resource Client:

definitions.py

import featureform as ff
from featureform import ResourceClient

rc = ResourceClient("localhost:8000")

# example query:
redis = client.get_provider("redis-quickstart")

Source code in src/featureform/register.py

class ResourceClient:
    """
    The resource client is used to retrieve information on specific resources
    (entities, providers, features, labels, training sets, models, users).

    Args:
        host (str): The hostname of the Featureform instance.
        insecure (bool): True if connecting to an insecure Featureform endpoint. False if using a self-signed or public TLS certificate
        cert_path (str): The path to a public certificate if using a self-signed certificate.

    **Using the Resource Client:**
    ``` py title="definitions.py"
    import featureform as ff
    from featureform import ResourceClient

    rc = ResourceClient("localhost:8000")

    # example query:
    redis = client.get_provider("redis-quickstart")
    ```
    """

    def __init__(
        self, host=None, local=False, insecure=False, cert_path=None, dry_run=False
    ):
        if local:
            raise Exception(
                "Local mode is not supported in this version. Use featureform <= 1.12.0 for localmode"
            )

        # This line ensures that the warning is only raised if ResourceClient is instantiated directly
        # TODO: Remove this check once ServingClient is deprecated
        is_instantiated_directed = inspect.stack()[1].function != "__init__"
        if is_instantiated_directed:
            warnings.warn(
                "ResourceClient is deprecated and will be removed in future versions; use Client instead.",
                PendingDeprecationWarning,
            )
        self._dry_run = dry_run
        self._stub = None
        self.local = local

        if dry_run:
            return

        host = host or os.getenv("FEATUREFORM_HOST")
        if host is None:
            raise RuntimeError(
                "If not in local mode then `host` must be passed or the environment"
                " variable FEATUREFORM_HOST must be set."
            )
        if insecure:
            channel = insecure_channel(host)
        else:
            channel = secure_channel(host, cert_path)
        self._stub = GrpcClient(ff_grpc.ApiStub(channel))
        self._host = host

    def apply(self, asynchronous=False, verbose=False):
        """
        Apply all definitions, creating and retrieving all specified resources.

        ```python
        import featureform as ff
        client = ff.Client()

        ff.register_postgres(
            host="localhost",
            port=5432,
        )

        client.apply()
        ```

        Args:
            asynchronous (bool): If True, apply will return immediately and not wait for resources to be created. If False, apply will wait for resources to be created and print out the status of each resource.

        """

        try:
            resource_state = state()
            if resource_state.is_empty():
                print("No resources to apply")
                return

            print(f"Applying Run: {get_run()}")

            if self._dry_run:
                print(resource_state.sorted_list())
                return

            resource_state.create_all(
                self._stub, global_registrar.get_client_objects_for_resource()
            )

            if not asynchronous and self._stub:
                resources = resource_state.sorted_list()
                display_statuses(self._stub, resources, verbose=verbose)
        finally:
            if feature_flag.is_enabled("FF_GET_EQUIVALENT_VARIANTS", True):
                set_run("")
            clear_state()

    def get_user(self, name, local=False):
        """Get a user. Prints out name of user, and all resources associated with the user.

        **Examples:**

        ``` py title="Input"
        featureformer = rc.get_user("featureformer")
        ```

        ``` json title="Output"
        // get_user prints out formatted information on user
        USER NAME:                     featureformer
        -----------------------------------------------

        NAME                           VARIANT                        TYPE
        avg_transactions               quickstart                     feature
        fraudulent                     quickstart                     label
        fraud_training                 quickstart                     training set
        transactions                   kaggle                         source
        average_user_transaction       quickstart                     source
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(featureformer)
        ```

        ``` json title="Output"
        // get_user returns the User object

        name: "featureformer"
        features {
        name: "avg_transactions"
        variant: "quickstart"
        }
        labels {
        name: "fraudulent"
        variant: "quickstart"
        }
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        sources {
        name: "transactions"
        variant: "kaggle"
        }
        sources {
        name: "average_user_transaction"
        variant: "quickstart"
        }
        ```

        Args:
            name (str): Name of user to be retrieved

        Returns:
            user (User): User
        """
        return get_user_info(self._stub, name)

    def get_entity(self, name, local=False):
        """Get an entity. Prints out information on entity, and all resources associated with the entity.

        **Examples:**

        ``` py title="Input"
        entity = rc.get_entity("user")
        ```

        ``` json title="Output"
        // get_entity prints out formatted information on entity

        ENTITY NAME:                   user
        STATUS:                        NO_STATUS
        -----------------------------------------------

        NAME                           VARIANT                        TYPE
        avg_transactions               quickstart                     feature
        fraudulent                     quickstart                     label
        fraud_training                 quickstart                     training set
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(postgres)
        ```

        ``` json title="Output"
        // get_entity returns the Entity object

        name: "user"
        features {
            name: "avg_transactions"
            variant: "quickstart"
        }
        labels {
            name: "fraudulent"
            variant: "quickstart"
        }
        trainingsets {
            name: "fraud_training"
            variant: "quickstart"
        }
        ```
        """
        return get_entity_info(self._stub, name)

    def get_model(self, name, local=False) -> Model:
        """Get a model. Prints out information on model, and all resources associated with the model.

        Args:
            name (str): Name of model to be retrieved

        Returns:
            model (Model): Model
        """
        model = None
        model_proto = get_resource_info(self._stub, "model", name)
        if model_proto is not None:
            model = Model(model_proto.name, description="", tags=[], properties={})

        return model

    def get_provider(self, name, local=False):
        """Get a provider. Prints out information on provider, and all resources associated with the provider.

        **Examples:**

        ``` py title="Input"
        postgres = client.get_provider("postgres-quickstart")
        ```

        ``` json title="Output"
        // get_provider prints out formatted information on provider

        NAME:                          postgres-quickstart
        DESCRIPTION:                   A Postgres deployment we created for the Featureform quickstart
        TYPE:                          POSTGRES_OFFLINE
        SOFTWARE:                      postgres
        STATUS:                        NO_STATUS
        -----------------------------------------------
        SOURCES:
        NAME                           VARIANT
        transactions                   kaggle
        average_user_transaction       quickstart
        -----------------------------------------------
        FEATURES:
        NAME                           VARIANT
        -----------------------------------------------
        LABELS:
        NAME                           VARIANT
        fraudulent                     quickstart
        -----------------------------------------------
        TRAINING SETS:
        NAME                           VARIANT
        fraud_training                 quickstart
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(postgres)
        ```

        ``` json title="Output"
        // get_provider returns the Provider object

        name: "postgres-quickstart"
        description: "A Postgres deployment we created for the Featureform quickstart"
        type: "POSTGRES_OFFLINE"
        software: "postgres"
        serialized_config: "{\"Host\": \"quickstart-postgres\",
                            \"Port\": \"5432\",
                            \"Username\": \"postgres\",
                            \"Password\": \"password\",
                            \"Database\": \"postgres\"}"
        sources {
        name: "transactions"
        variant: "kaggle"
        }
        sources {
        name: "average_user_transaction"
        variant: "quickstart"
        }
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        labels {
        name: "fraudulent"
        variant: "quickstart"
        }
        ```

        Args:
            name (str): Name of provider to be retrieved

        Returns:
            provider (Provider): Provider
        """
        return get_provider_info(self._stub, name)

    def get_feature(self, name, variant):
        name_variant = metadata_pb2.NameVariant(name=name, variant=variant)
        feature = None
        for x in self._stub.GetFeatureVariants(iter([name_variant])):
            feature = x
            break

        return FeatureVariant(
            created=None,
            name=feature.name,
            variant=feature.variant,
            source=(feature.source.name, feature.source.variant),
            value_type=feature.type,
            entity=feature.entity,
            owner=feature.owner,
            provider=feature.provider,
            location=ResourceColumnMapping("", "", ""),
            description=feature.description,
            status=feature.status.Status._enum_type.values[feature.status.status].name,
        )

    def print_feature(self, name, variant=None, local=False):
        """Get a feature. Prints out information on feature, and all variants associated with the feature. If variant is included, print information on that specific variant and all resources associated with it.

        **Examples:**

        ``` py title="Input"
        avg_transactions = rc.get_feature("avg_transactions")
        ```

        ``` json title="Output"
        // get_feature prints out formatted information on feature

        NAME:                          avg_transactions
        STATUS:                        NO_STATUS
        -----------------------------------------------
        VARIANTS:
        quickstart                     default
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(avg_transactions)
        ```

        ``` json title="Output"
        // get_feature returns the Feature object

        name: "avg_transactions"
        default_variant: "quickstart"
        variants: "quickstart"
        ```

        ``` py title="Input"
        avg_transactions_variant = ff.get_feature("avg_transactions", "quickstart")
        ```

        ``` json title="Output"
        // get_feature with variant provided prints out formatted information on feature variant

        NAME:                          avg_transactions
        VARIANT:                       quickstart
        TYPE:                          float32
        ENTITY:                        user
        OWNER:                         featureformer
        PROVIDER:                      redis-quickstart
        STATUS:                        NO_STATUS
        -----------------------------------------------
        SOURCE:
        NAME                           VARIANT
        average_user_transaction       quickstart
        -----------------------------------------------
        TRAINING SETS:
        NAME                           VARIANT
        fraud_training                 quickstart
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(avg_transactions_variant)
        ```

        ``` json title="Output"
        // get_feature returns the FeatureVariant object

        name: "avg_transactions"
        variant: "quickstart"
        source {
        name: "average_user_transaction"
        variant: "quickstart"
        }
        type: "float32"
        entity: "user"
        created {
        seconds: 1658168552
        nanos: 142461900
        }
        owner: "featureformer"
        provider: "redis-quickstart"
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        columns {
        entity: "user_id"
        value: "avg_transaction_amt"
        }
        ```

        Args:
            name (str): Name of feature to be retrieved
            variant (str): Name of variant of feature

        Returns:
            feature (Union[Feature, FeatureVariant]): Feature or FeatureVariant
        """
        if not variant:
            return get_resource_info(self._stub, "feature", name)
        return get_feature_variant_info(self._stub, name, variant)

    def get_label(self, name, variant):
        name_variant = metadata_pb2.NameVariant(name=name, variant=variant)
        label = None
        for x in self._stub.GetLabelVariants(iter([name_variant])):
            label = x
            break

        return LabelVariant(
            name=label.name,
            variant=label.variant,
            source=(label.source.name, label.source.variant),
            value_type=label.type,
            entity=label.entity,
            owner=label.owner,
            provider=label.provider,
            location=ResourceColumnMapping("", "", ""),
            description=label.description,
            status=label.status.Status._enum_type.values[label.status.status].name,
        )

    def print_label(self, name, variant=None, local=False):
        """Get a label. Prints out information on label, and all variants associated with the label. If variant is included, print information on that specific variant and all resources associated with it.

        **Examples:**

        ``` py title="Input"
        fraudulent = rc.get_label("fraudulent")
        ```

        ``` json title="Output"
        // get_label prints out formatted information on label

        NAME:                          fraudulent
        STATUS:                        NO_STATUS
        -----------------------------------------------
        VARIANTS:
        quickstart                     default
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(fraudulent)
        ```

        ``` json title="Output"
        // get_label returns the Label object

        name: "fraudulent"
        default_variant: "quickstart"
        variants: "quickstart"
        ```

        ``` py title="Input"
        fraudulent_variant = ff.get_label("fraudulent", "quickstart")
        ```

        ``` json title="Output"
        // get_label with variant provided prints out formatted information on label variant

        NAME:                          fraudulent
        VARIANT:                       quickstart
        TYPE:                          bool
        ENTITY:                        user
        OWNER:                         featureformer
        PROVIDER:                      postgres-quickstart
        STATUS:                        NO_STATUS
        -----------------------------------------------
        SOURCE:
        NAME                           VARIANT
        transactions                   kaggle
        -----------------------------------------------
        TRAINING SETS:
        NAME                           VARIANT
        fraud_training                 quickstart
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(fraudulent_variant)
        ```

        ``` json title="Output"
        // get_label returns the LabelVariant object

        name: "fraudulent"
        variant: "quickstart"
        type: "bool"
        source {
        name: "transactions"
        variant: "kaggle"
        }
        entity: "user"
        created {
        seconds: 1658168552
        nanos: 154924300
        }
        owner: "featureformer"
        provider: "postgres-quickstart"
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        columns {
        entity: "customerid"
        value: "isfraud"
        }
        ```

        Args:
            name (str): Name of label to be retrieved
            variant (str): Name of variant of label

        Returns:
            label (Union[label, LabelVariant]): Label or LabelVariant
        """
        if not variant:
            return get_resource_info(self._stub, "label", name)
        return get_label_variant_info(self._stub, name, variant)

    def get_training_set(self, name, variant):
        name_variant = metadata_pb2.NameVariant(name=name, variant=variant)
        ts = None
        for x in self._stub.GetTrainingSetVariants(iter([name_variant])):
            ts = x
            break

        return TrainingSetVariant(
            created=None,
            name=ts.name,
            variant=ts.variant,
            owner=ts.owner,
            description=ts.description,
            status=ts.status.Status._enum_type.values[ts.status.status].name,
            label=(ts.label.name, ts.label.variant),
            features=[(f.name, f.variant) for f in ts.features],
            feature_lags=[],
            provider=ts.provider,
            # TODO: apply values from proto
            tags=[],
            properties={},
        )

    def print_training_set(self, name, variant=None, local=False):
        """Get a training set. Prints out information on training set, and all variants associated with the training set. If variant is included, print information on that specific variant and all resources associated with it.

        **Examples:**

        ``` py title="Input"
        fraud_training = rc.get_training_set("fraud_training")
        ```

        ``` json title="Output"
        // get_training_set prints out formatted information on training set

        NAME:                          fraud_training
        STATUS:                        NO_STATUS
        -----------------------------------------------
        VARIANTS:
        quickstart                     default
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(fraud_training)
        ```

        ``` json title="Output"
        // get_training_set returns the TrainingSet object

        name: "fraud_training"
        default_variant: "quickstart"
        variants: "quickstart"
        ```

        ``` py title="Input"
        fraudulent_variant = ff.get_training set("fraudulent", "quickstart")
        ```

        ``` json title="Output"
        // get_training_set with variant provided prints out formatted information on training set variant

        NAME:                          fraud_training
        VARIANT:                       quickstart
        OWNER:                         featureformer
        PROVIDER:                      postgres-quickstart
        STATUS:                        NO_STATUS
        -----------------------------------------------
        LABEL:
        NAME                           VARIANT
        fraudulent                     quickstart
        -----------------------------------------------
        FEATURES:
        NAME                           VARIANT
        avg_transactions               quickstart
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(fraudulent_variant)
        ```

        ``` json title="Output"
        // get_training_set returns the TrainingSetVariant object

        name: "fraud_training"
        variant: "quickstart"
        owner: "featureformer"
        created {
        seconds: 1658168552
        nanos: 157934800
        }
        provider: "postgres-quickstart"
        features {
        name: "avg_transactions"
        variant: "quickstart"
        }
        label {
        name: "fraudulent"
        variant: "quickstart"
        }
        ```

        Args:
            name (str): Name of training set to be retrieved
            variant (str): Name of variant of training set

        Returns:
            training_set (Union[TrainingSet, TrainingSetVariant]): TrainingSet or TrainingSetVariant
        """
        if not variant:
            return get_resource_info(self._stub, "training-set", name)
        return get_training_set_variant_info(self._stub, name, variant)

    def get_source(self, name, variant):
        name_variant = metadata_pb2.NameVariantRequest(
            name_variant=metadata_pb2.NameVariant(name=name, variant=variant)
        )
        source = None
        for x in self._stub.GetSourceVariants(iter([name_variant])):
            source = x
            break

        definition = self._get_source_definition(source)

        source_variant = SourceVariant(
            created=None,
            name=source.name,
            definition=definition,
            owner=source.owner,
            provider=source.provider,
            description=source.description,
            variant=source.variant,
            status=source.status.Status._enum_type.values[source.status.status].name,
            tags=[],
            properties={},
            source_text=(
                definition.source_text if type(definition) == DFTransformation else ""
            ),
        )
        return ColumnSourceRegistrar(self, source_variant)

    def _get_source_definition(self, source):
        if source.primaryData.table.name:
            return PrimaryData(SQLTable(source.primaryData.table.name))
        elif source.transformation:
            return self._get_transformation_definition(source)
        else:
            raise Exception(f"Invalid source type {source}")

    def _get_transformation_definition(self, source):
        if source.transformation.DFTransformation.query != bytes():
            transformation = source.transformation.DFTransformation
            return DFTransformation(
                query=transformation.query,
                inputs=[(input.name, input.variant) for input in transformation.inputs],
                source_text=transformation.source_text,
            )
        elif source.transformation.SQLTransformation.query != "":
            return SQLTransformation(source.transformation.SQLTransformation.query)
        else:
            raise Exception(f"Invalid transformation type {source}")

    def print_source(self, name, variant=None, local=False):
        """Get a source. Prints out information on source, and all variants associated with the source. If variant is included, print information on that specific variant and all resources associated with it.

        **Examples:**

        ``` py title="Input"
        transactions = rc.get_transactions("transactions")
        ```

        ``` json title="Output"
        // get_source prints out formatted information on source

        NAME:                          transactions
        STATUS:                        NO_STATUS
        -----------------------------------------------
        VARIANTS:
        kaggle                         default
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(transactions)
        ```

        ``` json title="Output"
        // get_source returns the Source object

        name: "transactions"
        default_variant: "kaggle"
        variants: "kaggle"
        ```

        ``` py title="Input"
        transactions_variant = rc.get_source("transactions", "kaggle")
        ```

        ``` json title="Output"
        // get_source with variant provided prints out formatted information on source variant

        NAME:                          transactions
        VARIANT:                       kaggle
        OWNER:                         featureformer
        DESCRIPTION:                   Fraud Dataset From Kaggle
        PROVIDER:                      postgres-quickstart
        STATUS:                        NO_STATUS
        -----------------------------------------------
        DEFINITION:
        TRANSFORMATION

        -----------------------------------------------
        SOURCES
        NAME                           VARIANT
        -----------------------------------------------
        PRIMARY DATA
        Transactions
        FEATURES:
        NAME                           VARIANT
        -----------------------------------------------
        LABELS:
        NAME                           VARIANT
        fraudulent                     quickstart
        -----------------------------------------------
        TRAINING SETS:
        NAME                           VARIANT
        fraud_training                 quickstart
        -----------------------------------------------
        ```

        ``` py title="Input"
        print(transactions_variant)
        ```

        ``` json title="Output"
        // get_source returns the SourceVariant object

        name: "transactions"
        variant: "kaggle"
        owner: "featureformer"
        description: "Fraud Dataset From Kaggle"
        provider: "postgres-quickstart"
        created {
        seconds: 1658168552
        nanos: 128768000
        }
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        labels {
        name: "fraudulent"
        variant: "quickstart"
        }
        primaryData {
        table {
            name: "Transactions"
        }
        }
        ```

        Args:
            name (str): Name of source to be retrieved
            variant (str): Name of variant of source

        Returns:
            source (Union[Source, SourceVariant]): Source or SourceVariant
        """
        if not variant:
            return get_resource_info(self._stub, "source", name)
        return get_source_variant_info(self._stub, name, variant)

    def list_features(self, local=False):
        """List all features.

        **Examples:**
        ``` py title="Input"
        features_list = rc.list_features()
        ```

        ``` json title="Output"
        // list_features prints out formatted information on all features

        NAME                           VARIANT                        STATUS
        user_age                       quickstart (default)           READY
        avg_transactions               quickstart (default)           READY
        avg_transactions               production                     CREATED
        ```

        ``` py title="Input"
        print(features_list)
        ```

        ``` json title="Output"
        // list_features returns a list of Feature objects

        [name: "user_age"
        default_variant: "quickstart"
        variants: "quickstart"
        , name: "avg_transactions"
        default_variant: "quickstart"
        variants: "quickstart"
        variants: "production"
        ]
        ```

        Returns:
            features (List[Feature]): List of Feature Objects
        """
        return list_name_variant_status(self._stub, "feature")

    def list_labels(self, local=False):
        """List all labels.

        **Examples:**
        ``` py title="Input"
        features_list = rc.list_labels()
        ```

        ``` json title="Output"
        // list_labels prints out formatted information on all labels

        NAME                           VARIANT                        STATUS
        user_age                       quickstart (default)           READY
        avg_transactions               quickstart (default)           READY
        avg_transactions               production                     CREATED
        ```

        ``` py title="Input"
        print(label_list)
        ```

        ``` json title="Output"
        // list_features returns a list of Feature objects

        [name: "user_age"
        default_variant: "quickstart"
        variants: "quickstart"
        , name: "avg_transactions"
        default_variant: "quickstart"
        variants: "quickstart"
        variants: "production"
        ]
        ```

        Returns:
            labels (List[Label]): List of Label Objects
        """
        return list_name_variant_status(self._stub, "label")

    def list_users(self, local=False):
        """List all users. Prints a list of all users.

        **Examples:**
        ``` py title="Input"
        users_list = rc.list_users()
        ```

        ``` json title="Output"
        // list_users prints out formatted information on all users

        NAME                           STATUS
        featureformer                  NO_STATUS
        featureformers_friend          CREATED
        ```

        ``` py title="Input"
        print(features_list)
        ```

        ``` json title="Output"
        // list_features returns a list of Feature objects

        [name: "featureformer"
        features {
        name: "avg_transactions"
        variant: "quickstart"
        }
        labels {
        name: "fraudulent"
        variant: "quickstart"
        }
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        sources {
        name: "transactions"
        variant: "kaggle"
        }
        sources {
        name: "average_user_transaction"
        variant: "quickstart"
        },
        name: "featureformers_friend"
        features {
        name: "user_age"
        variant: "production"
        }
        sources {
        name: "user_profiles"
        variant: "production"
        }
        ]
        ```

        Returns:
            users (List[User]): List of User Objects
        """
        return list_name_status(self._stub, "user")

    def list_entities(self, local=False):
        """List all entities. Prints a list of all entities.

        **Examples:**
        ``` py title="Input"
        entities = rc.list_entities()
        ```

        ``` json title="Output"
        // list_entities prints out formatted information on all entities

        NAME                           STATUS
        user                           CREATED
        transaction                    CREATED
        ```

        ``` py title="Input"
        print(features_list)
        ```

        ``` json title="Output"
        // list_entities returns a list of Entity objects

        [name: "user"
        features {
        name: "avg_transactions"
        variant: "quickstart"
        }
        features {
        name: "avg_transactions"
        variant: "production"
        }
        features {
        name: "user_age"
        variant: "quickstart"
        }
        labels {
        name: "fraudulent"
        variant: "quickstart"
        }
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        ,
        name: "transaction"
        features {
        name: "amount_spent"
        variant: "production"
        }
        ]
        ```

        Returns:
            entities (List[Entity]): List of Entity Objects
        """
        return list_name_status(self._stub, "entity")

    def list_sources(self, local=False):
        """List all sources. Prints a list of all sources.

        **Examples:**
        ``` py title="Input"
        sources_list = rc.list_sources()
        ```

        ``` json title="Output"
        // list_sources prints out formatted information on all sources

        NAME                           VARIANT                        STATUS                         DESCRIPTION
        average_user_transaction       quickstart (default)           NO_STATUS                      the average transaction amount for a user
        transactions                   kaggle (default)               NO_STATUS                      Fraud Dataset From Kaggle
        ```

        ``` py title="Input"
        print(sources_list)
        ```

        ``` json title="Output"
        // list_sources returns a list of Source objects

        [name: "average_user_transaction"
        default_variant: "quickstart"
        variants: "quickstart"
        , name: "transactions"
        default_variant: "kaggle"
        variants: "kaggle"
        ]
        ```

        Returns:
            sources (List[Source]): List of Source Objects
        """
        return list_name_variant_status_desc(self._stub, "source")

    def list_training_sets(self, local=False):
        """List all training sets. Prints a list of all training sets.

        **Examples:**
        ``` py title="Input"
        training_sets_list = rc.list_training_sets()
        ```

        ``` json title="Output"
        // list_training_sets prints out formatted information on all training sets

        NAME                           VARIANT                        STATUS                         DESCRIPTION
        fraud_training                 quickstart (default)           READY                          Training set for fraud detection.
        fraud_training                 v2                             CREATED                        Improved training set for fraud detection.
        recommender                    v1 (default)                   CREATED                        Training set for recommender system.
        ```

        ``` py title="Input"
        print(training_sets_list)
        ```

        ``` json title="Output"
        // list_training_sets returns a list of TrainingSet objects

        [name: "fraud_training"
        default_variant: "quickstart"
        variants: "quickstart", "v2",
        name: "recommender"
        default_variant: "v1"
        variants: "v1"
        ]
        ```

        Returns:
            training_sets (List[TrainingSet]): List of TrainingSet Objects
        """
        return list_name_variant_status_desc(self._stub, "training-set")

    def list_models(self, local=False) -> List[Model]:
        """List all models. Prints a list of all models.

        Returns:
            models (List[Model]): List of Model Objects
        """
        model_protos = list_name(self._stub, "model")
        # TODO: apply values from proto
        models = [Model(proto.name, tags=[], properties={}) for proto in model_protos]

        return models

    def list_providers(self, local=False):
        """List all providers. Prints a list of all providers.

        **Examples:**
        ``` py title="Input"
        providers_list = rc.list_providers()
        ```

        ``` json title="Output"
        // list_providers prints out formatted information on all providers

        NAME                           STATUS                         DESCRIPTION
        redis-quickstart               CREATED                      A Redis deployment we created for the Featureform quickstart
        postgres-quickstart            CREATED                      A Postgres deployment we created for the Featureform quickst
        ```

        ``` py title="Input"
        print(providers_list)
        ```

        ``` json title="Output"
        // list_providers returns a list of Providers objects

        [name: "redis-quickstart"
        description: "A Redis deployment we created for the Featureform quickstart"
        type: "REDIS_ONLINE"
        software: "redis"
        serialized_config: "{\"Addr\": \"quickstart-redis:6379\", \"Password\": \"\", \"DB\": 0}"
        features {
        name: "avg_transactions"
        variant: "quickstart"
        }
        features {
        name: "avg_transactions"
        variant: "production"
        }
        features {
        name: "user_age"
        variant: "quickstart"
        }
        , name: "postgres-quickstart"
        description: "A Postgres deployment we created for the Featureform quickstart"
        type: "POSTGRES_OFFLINE"
        software: "postgres"
        serialized_config: "{\"Host\": \"quickstart-postgres\", \"Port\": \"5432\", \"Username\": \"postgres\", \"Password\": \"password\", \"Database\": \"postgres\"}"
        sources {
        name: "transactions"
        variant: "kaggle"
        }
        sources {
        name: "average_user_transaction"
        variant: "quickstart"
        }
        trainingsets {
        name: "fraud_training"
        variant: "quickstart"
        }
        labels {
        name: "fraudulent"
        variant: "quickstart"
        }
        ]
        ```

        Returns:
            providers (List[Provider]): List of Provider Objects
        """
        return list_name_status_desc(self._stub, "provider")

    def search(self, raw_query, local=False):
        """Search for registered resources. Prints a list of results.

        **Examples:**
        ``` py title="Input"
        providers_list = rc.search("transact")
        ```

        ``` json title="Output"
        // search prints out formatted information on all matches

        NAME                           VARIANT            TYPE
        avg_transactions               default            Source
        ```
        """
        if type(raw_query) != str or len(raw_query) == 0:
            raise Exception("query must be string and cannot be empty")
        processed_query = raw_query.translate({ord(i): None for i in ".,-@!*#"})
        return search(processed_query, self._host)

apply(asynchronous=False, verbose=False)

Apply all definitions, creating and retrieving all specified resources.

import featureform as ff
client = ff.Client()

ff.register_postgres(
    host="localhost",
    port=5432,
)

client.apply()

Parameters:

Name	Type	Description	Default
asynchronous	bool	If True, apply will return immediately and not wait for resources to be created. If False, apply will wait for resources to be created and print out the status of each resource.	False

Source code in src/featureform/register.py

def apply(self, asynchronous=False, verbose=False):
    """
    Apply all definitions, creating and retrieving all specified resources.

    ```python
    import featureform as ff
    client = ff.Client()

    ff.register_postgres(
        host="localhost",
        port=5432,
    )

    client.apply()
    ```

    Args:
        asynchronous (bool): If True, apply will return immediately and not wait for resources to be created. If False, apply will wait for resources to be created and print out the status of each resource.

    """

    try:
        resource_state = state()
        if resource_state.is_empty():
            print("No resources to apply")
            return

        print(f"Applying Run: {get_run()}")

        if self._dry_run:
            print(resource_state.sorted_list())
            return

        resource_state.create_all(
            self._stub, global_registrar.get_client_objects_for_resource()
        )

        if not asynchronous and self._stub:
            resources = resource_state.sorted_list()
            display_statuses(self._stub, resources, verbose=verbose)
    finally:
        if feature_flag.is_enabled("FF_GET_EQUIVALENT_VARIANTS", True):
            set_run("")
        clear_state()

get_entity(name, local=False)

Get an entity. Prints out information on entity, and all resources associated with the entity.

Examples:

Input

entity = rc.get_entity("user")

Output

// get_entity prints out formatted information on entity

ENTITY NAME:                   user
STATUS:                        NO_STATUS
-----------------------------------------------

NAME                           VARIANT                        TYPE
avg_transactions               quickstart                     feature
fraudulent                     quickstart                     label
fraud_training                 quickstart                     training set
-----------------------------------------------

Input

print(postgres)

Output

// get_entity returns the Entity object

name: "user"
features {
    name: "avg_transactions"
    variant: "quickstart"
}
labels {
    name: "fraudulent"
    variant: "quickstart"
}
trainingsets {
    name: "fraud_training"
    variant: "quickstart"
}

Source code in src/featureform/register.py

def get_entity(self, name, local=False):
    """Get an entity. Prints out information on entity, and all resources associated with the entity.

    **Examples:**

    ``` py title="Input"
    entity = rc.get_entity("user")
    ```

    ``` json title="Output"
    // get_entity prints out formatted information on entity

    ENTITY NAME:                   user
    STATUS:                        NO_STATUS
    -----------------------------------------------

    NAME                           VARIANT                        TYPE
    avg_transactions               quickstart                     feature
    fraudulent                     quickstart                     label
    fraud_training                 quickstart                     training set
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(postgres)
    ```

    ``` json title="Output"
    // get_entity returns the Entity object

    name: "user"
    features {
        name: "avg_transactions"
        variant: "quickstart"
    }
    labels {
        name: "fraudulent"
        variant: "quickstart"
    }
    trainingsets {
        name: "fraud_training"
        variant: "quickstart"
    }
    ```
    """
    return get_entity_info(self._stub, name)

get_model(name, local=False)

Get a model. Prints out information on model, and all resources associated with the model.

Parameters:

Name	Type	Description	Default
name	str	Name of model to be retrieved	required

Returns:

Name	Type	Description
model	Model	Model

Source code in src/featureform/register.py

def get_model(self, name, local=False) -> Model:
    """Get a model. Prints out information on model, and all resources associated with the model.

    Args:
        name (str): Name of model to be retrieved

    Returns:
        model (Model): Model
    """
    model = None
    model_proto = get_resource_info(self._stub, "model", name)
    if model_proto is not None:
        model = Model(model_proto.name, description="", tags=[], properties={})

    return model

get_provider(name, local=False)

Get a provider. Prints out information on provider, and all resources associated with the provider.

Examples:

Input

postgres = client.get_provider("postgres-quickstart")

Output

// get_provider prints out formatted information on provider

NAME:                          postgres-quickstart
DESCRIPTION:                   A Postgres deployment we created for the Featureform quickstart
TYPE:                          POSTGRES_OFFLINE
SOFTWARE:                      postgres
STATUS:                        NO_STATUS
-----------------------------------------------
SOURCES:
NAME                           VARIANT
transactions                   kaggle
average_user_transaction       quickstart
-----------------------------------------------
FEATURES:
NAME                           VARIANT
-----------------------------------------------
LABELS:
NAME                           VARIANT
fraudulent                     quickstart
-----------------------------------------------
TRAINING SETS:
NAME                           VARIANT
fraud_training                 quickstart
-----------------------------------------------

Input

print(postgres)

Output

// get_provider returns the Provider object

name: "postgres-quickstart"
description: "A Postgres deployment we created for the Featureform quickstart"
type: "POSTGRES_OFFLINE"
software: "postgres"
serialized_config: "{"Host": "quickstart-postgres",
                    "Port": "5432",
                    "Username": "postgres",
                    "Password": "password",
                    "Database": "postgres"}"
sources {
name: "transactions"
variant: "kaggle"
}
sources {
name: "average_user_transaction"
variant: "quickstart"
}
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
labels {
name: "fraudulent"
variant: "quickstart"
}

Parameters:

Name	Type	Description	Default
name	str	Name of provider to be retrieved	required

Returns:

Name	Type	Description
provider	Provider	Provider

Source code in src/featureform/register.py

def get_provider(self, name, local=False):
    """Get a provider. Prints out information on provider, and all resources associated with the provider.

    **Examples:**

    ``` py title="Input"
    postgres = client.get_provider("postgres-quickstart")
    ```

    ``` json title="Output"
    // get_provider prints out formatted information on provider

    NAME:                          postgres-quickstart
    DESCRIPTION:                   A Postgres deployment we created for the Featureform quickstart
    TYPE:                          POSTGRES_OFFLINE
    SOFTWARE:                      postgres
    STATUS:                        NO_STATUS
    -----------------------------------------------
    SOURCES:
    NAME                           VARIANT
    transactions                   kaggle
    average_user_transaction       quickstart
    -----------------------------------------------
    FEATURES:
    NAME                           VARIANT
    -----------------------------------------------
    LABELS:
    NAME                           VARIANT
    fraudulent                     quickstart
    -----------------------------------------------
    TRAINING SETS:
    NAME                           VARIANT
    fraud_training                 quickstart
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(postgres)
    ```

    ``` json title="Output"
    // get_provider returns the Provider object

    name: "postgres-quickstart"
    description: "A Postgres deployment we created for the Featureform quickstart"
    type: "POSTGRES_OFFLINE"
    software: "postgres"
    serialized_config: "{\"Host\": \"quickstart-postgres\",
                        \"Port\": \"5432\",
                        \"Username\": \"postgres\",
                        \"Password\": \"password\",
                        \"Database\": \"postgres\"}"
    sources {
    name: "transactions"
    variant: "kaggle"
    }
    sources {
    name: "average_user_transaction"
    variant: "quickstart"
    }
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    labels {
    name: "fraudulent"
    variant: "quickstart"
    }
    ```

    Args:
        name (str): Name of provider to be retrieved

    Returns:
        provider (Provider): Provider
    """
    return get_provider_info(self._stub, name)

get_user(name, local=False)

Get a user. Prints out name of user, and all resources associated with the user.

Examples:

Input

featureformer = rc.get_user("featureformer")

Output

// get_user prints out formatted information on user
USER NAME:                     featureformer
-----------------------------------------------

NAME                           VARIANT                        TYPE
avg_transactions               quickstart                     feature
fraudulent                     quickstart                     label
fraud_training                 quickstart                     training set
transactions                   kaggle                         source
average_user_transaction       quickstart                     source
-----------------------------------------------

Input

print(featureformer)

Output

// get_user returns the User object

name: "featureformer"
features {
name: "avg_transactions"
variant: "quickstart"
}
labels {
name: "fraudulent"
variant: "quickstart"
}
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
sources {
name: "transactions"
variant: "kaggle"
}
sources {
name: "average_user_transaction"
variant: "quickstart"
}

Parameters:

Name	Type	Description	Default
name	str	Name of user to be retrieved	required

Returns:

Name	Type	Description
user	User	User

Source code in src/featureform/register.py

def get_user(self, name, local=False):
    """Get a user. Prints out name of user, and all resources associated with the user.

    **Examples:**

    ``` py title="Input"
    featureformer = rc.get_user("featureformer")
    ```

    ``` json title="Output"
    // get_user prints out formatted information on user
    USER NAME:                     featureformer
    -----------------------------------------------

    NAME                           VARIANT                        TYPE
    avg_transactions               quickstart                     feature
    fraudulent                     quickstart                     label
    fraud_training                 quickstart                     training set
    transactions                   kaggle                         source
    average_user_transaction       quickstart                     source
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(featureformer)
    ```

    ``` json title="Output"
    // get_user returns the User object

    name: "featureformer"
    features {
    name: "avg_transactions"
    variant: "quickstart"
    }
    labels {
    name: "fraudulent"
    variant: "quickstart"
    }
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    sources {
    name: "transactions"
    variant: "kaggle"
    }
    sources {
    name: "average_user_transaction"
    variant: "quickstart"
    }
    ```

    Args:
        name (str): Name of user to be retrieved

    Returns:
        user (User): User
    """
    return get_user_info(self._stub, name)

list_entities(local=False)

List all entities. Prints a list of all entities.

Examples:

Input

entities = rc.list_entities()

Output

// list_entities prints out formatted information on all entities

NAME                           STATUS
user                           CREATED
transaction                    CREATED

Input

print(features_list)

Output

// list_entities returns a list of Entity objects

[name: "user"
features {
name: "avg_transactions"
variant: "quickstart"
}
features {
name: "avg_transactions"
variant: "production"
}
features {
name: "user_age"
variant: "quickstart"
}
labels {
name: "fraudulent"
variant: "quickstart"
}
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
,
name: "transaction"
features {
name: "amount_spent"
variant: "production"
}
]

Returns:

Name	Type	Description
entities	List[Entity]	List of Entity Objects

Source code in src/featureform/register.py

def list_entities(self, local=False):
    """List all entities. Prints a list of all entities.

    **Examples:**
    ``` py title="Input"
    entities = rc.list_entities()
    ```

    ``` json title="Output"
    // list_entities prints out formatted information on all entities

    NAME                           STATUS
    user                           CREATED
    transaction                    CREATED
    ```

    ``` py title="Input"
    print(features_list)
    ```

    ``` json title="Output"
    // list_entities returns a list of Entity objects

    [name: "user"
    features {
    name: "avg_transactions"
    variant: "quickstart"
    }
    features {
    name: "avg_transactions"
    variant: "production"
    }
    features {
    name: "user_age"
    variant: "quickstart"
    }
    labels {
    name: "fraudulent"
    variant: "quickstart"
    }
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    ,
    name: "transaction"
    features {
    name: "amount_spent"
    variant: "production"
    }
    ]
    ```

    Returns:
        entities (List[Entity]): List of Entity Objects
    """
    return list_name_status(self._stub, "entity")

list_features(local=False)

List all features.

Examples:

Input

features_list = rc.list_features()

Output

// list_features prints out formatted information on all features

NAME                           VARIANT                        STATUS
user_age                       quickstart (default)           READY
avg_transactions               quickstart (default)           READY
avg_transactions               production                     CREATED

Input

print(features_list)

Output

// list_features returns a list of Feature objects

[name: "user_age"
default_variant: "quickstart"
variants: "quickstart"
, name: "avg_transactions"
default_variant: "quickstart"
variants: "quickstart"
variants: "production"
]

Returns:

Name	Type	Description
features	List[Feature]	List of Feature Objects

Source code in src/featureform/register.py

def list_features(self, local=False):
    """List all features.

    **Examples:**
    ``` py title="Input"
    features_list = rc.list_features()
    ```

    ``` json title="Output"
    // list_features prints out formatted information on all features

    NAME                           VARIANT                        STATUS
    user_age                       quickstart (default)           READY
    avg_transactions               quickstart (default)           READY
    avg_transactions               production                     CREATED
    ```

    ``` py title="Input"
    print(features_list)
    ```

    ``` json title="Output"
    // list_features returns a list of Feature objects

    [name: "user_age"
    default_variant: "quickstart"
    variants: "quickstart"
    , name: "avg_transactions"
    default_variant: "quickstart"
    variants: "quickstart"
    variants: "production"
    ]
    ```

    Returns:
        features (List[Feature]): List of Feature Objects
    """
    return list_name_variant_status(self._stub, "feature")

list_labels(local=False)

List all labels.

Examples:

Input

features_list = rc.list_labels()

Output

// list_labels prints out formatted information on all labels

NAME                           VARIANT                        STATUS
user_age                       quickstart (default)           READY
avg_transactions               quickstart (default)           READY
avg_transactions               production                     CREATED

Input

print(label_list)

Output

// list_features returns a list of Feature objects

[name: "user_age"
default_variant: "quickstart"
variants: "quickstart"
, name: "avg_transactions"
default_variant: "quickstart"
variants: "quickstart"
variants: "production"
]

Returns:

Name	Type	Description
labels	List[Label]	List of Label Objects

Source code in src/featureform/register.py

def list_labels(self, local=False):
    """List all labels.

    **Examples:**
    ``` py title="Input"
    features_list = rc.list_labels()
    ```

    ``` json title="Output"
    // list_labels prints out formatted information on all labels

    NAME                           VARIANT                        STATUS
    user_age                       quickstart (default)           READY
    avg_transactions               quickstart (default)           READY
    avg_transactions               production                     CREATED
    ```

    ``` py title="Input"
    print(label_list)
    ```

    ``` json title="Output"
    // list_features returns a list of Feature objects

    [name: "user_age"
    default_variant: "quickstart"
    variants: "quickstart"
    , name: "avg_transactions"
    default_variant: "quickstart"
    variants: "quickstart"
    variants: "production"
    ]
    ```

    Returns:
        labels (List[Label]): List of Label Objects
    """
    return list_name_variant_status(self._stub, "label")

list_models(local=False)

List all models. Prints a list of all models.

Returns:

Name	Type	Description
models	List[Model]	List of Model Objects

Source code in src/featureform/register.py

def list_models(self, local=False) -> List[Model]:
    """List all models. Prints a list of all models.

    Returns:
        models (List[Model]): List of Model Objects
    """
    model_protos = list_name(self._stub, "model")
    # TODO: apply values from proto
    models = [Model(proto.name, tags=[], properties={}) for proto in model_protos]

    return models

list_providers(local=False)

List all providers. Prints a list of all providers.

Examples:

Input

providers_list = rc.list_providers()

Output

// list_providers prints out formatted information on all providers

NAME                           STATUS                         DESCRIPTION
redis-quickstart               CREATED                      A Redis deployment we created for the Featureform quickstart
postgres-quickstart            CREATED                      A Postgres deployment we created for the Featureform quickst

Input

print(providers_list)

Output

// list_providers returns a list of Providers objects

[name: "redis-quickstart"
description: "A Redis deployment we created for the Featureform quickstart"
type: "REDIS_ONLINE"
software: "redis"
serialized_config: "{"Addr": "quickstart-redis:6379", "Password": "", "DB": 0}"
features {
name: "avg_transactions"
variant: "quickstart"
}
features {
name: "avg_transactions"
variant: "production"
}
features {
name: "user_age"
variant: "quickstart"
}
, name: "postgres-quickstart"
description: "A Postgres deployment we created for the Featureform quickstart"
type: "POSTGRES_OFFLINE"
software: "postgres"
serialized_config: "{"Host": "quickstart-postgres", "Port": "5432", "Username": "postgres", "Password": "password", "Database": "postgres"}"
sources {
name: "transactions"
variant: "kaggle"
}
sources {
name: "average_user_transaction"
variant: "quickstart"
}
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
labels {
name: "fraudulent"
variant: "quickstart"
}
]

Returns:

Name	Type	Description
providers	List[Provider]	List of Provider Objects

Source code in src/featureform/register.py

def list_providers(self, local=False):
    """List all providers. Prints a list of all providers.

    **Examples:**
    ``` py title="Input"
    providers_list = rc.list_providers()
    ```

    ``` json title="Output"
    // list_providers prints out formatted information on all providers

    NAME                           STATUS                         DESCRIPTION
    redis-quickstart               CREATED                      A Redis deployment we created for the Featureform quickstart
    postgres-quickstart            CREATED                      A Postgres deployment we created for the Featureform quickst
    ```

    ``` py title="Input"
    print(providers_list)
    ```

    ``` json title="Output"
    // list_providers returns a list of Providers objects

    [name: "redis-quickstart"
    description: "A Redis deployment we created for the Featureform quickstart"
    type: "REDIS_ONLINE"
    software: "redis"
    serialized_config: "{\"Addr\": \"quickstart-redis:6379\", \"Password\": \"\", \"DB\": 0}"
    features {
    name: "avg_transactions"
    variant: "quickstart"
    }
    features {
    name: "avg_transactions"
    variant: "production"
    }
    features {
    name: "user_age"
    variant: "quickstart"
    }
    , name: "postgres-quickstart"
    description: "A Postgres deployment we created for the Featureform quickstart"
    type: "POSTGRES_OFFLINE"
    software: "postgres"
    serialized_config: "{\"Host\": \"quickstart-postgres\", \"Port\": \"5432\", \"Username\": \"postgres\", \"Password\": \"password\", \"Database\": \"postgres\"}"
    sources {
    name: "transactions"
    variant: "kaggle"
    }
    sources {
    name: "average_user_transaction"
    variant: "quickstart"
    }
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    labels {
    name: "fraudulent"
    variant: "quickstart"
    }
    ]
    ```

    Returns:
        providers (List[Provider]): List of Provider Objects
    """
    return list_name_status_desc(self._stub, "provider")

list_sources(local=False)

List all sources. Prints a list of all sources.

Examples:

Input

sources_list = rc.list_sources()

Output

// list_sources prints out formatted information on all sources

NAME                           VARIANT                        STATUS                         DESCRIPTION
average_user_transaction       quickstart (default)           NO_STATUS                      the average transaction amount for a user
transactions                   kaggle (default)               NO_STATUS                      Fraud Dataset From Kaggle

Input

print(sources_list)

Output

// list_sources returns a list of Source objects

[name: "average_user_transaction"
default_variant: "quickstart"
variants: "quickstart"
, name: "transactions"
default_variant: "kaggle"
variants: "kaggle"
]

Returns:

Name	Type	Description
sources	List[Source]	List of Source Objects

Source code in src/featureform/register.py

def list_sources(self, local=False):
    """List all sources. Prints a list of all sources.

    **Examples:**
    ``` py title="Input"
    sources_list = rc.list_sources()
    ```

    ``` json title="Output"
    // list_sources prints out formatted information on all sources

    NAME                           VARIANT                        STATUS                         DESCRIPTION
    average_user_transaction       quickstart (default)           NO_STATUS                      the average transaction amount for a user
    transactions                   kaggle (default)               NO_STATUS                      Fraud Dataset From Kaggle
    ```

    ``` py title="Input"
    print(sources_list)
    ```

    ``` json title="Output"
    // list_sources returns a list of Source objects

    [name: "average_user_transaction"
    default_variant: "quickstart"
    variants: "quickstart"
    , name: "transactions"
    default_variant: "kaggle"
    variants: "kaggle"
    ]
    ```

    Returns:
        sources (List[Source]): List of Source Objects
    """
    return list_name_variant_status_desc(self._stub, "source")

list_training_sets(local=False)

List all training sets. Prints a list of all training sets.

Examples:

Input

training_sets_list = rc.list_training_sets()

Output

// list_training_sets prints out formatted information on all training sets

NAME                           VARIANT                        STATUS                         DESCRIPTION
fraud_training                 quickstart (default)           READY                          Training set for fraud detection.
fraud_training                 v2                             CREATED                        Improved training set for fraud detection.
recommender                    v1 (default)                   CREATED                        Training set for recommender system.

Input

print(training_sets_list)

Output

// list_training_sets returns a list of TrainingSet objects

[name: "fraud_training"
default_variant: "quickstart"
variants: "quickstart", "v2",
name: "recommender"
default_variant: "v1"
variants: "v1"
]

Returns:

Name	Type	Description
training_sets	List[TrainingSet]	List of TrainingSet Objects

Source code in src/featureform/register.py

def list_training_sets(self, local=False):
    """List all training sets. Prints a list of all training sets.

    **Examples:**
    ``` py title="Input"
    training_sets_list = rc.list_training_sets()
    ```

    ``` json title="Output"
    // list_training_sets prints out formatted information on all training sets

    NAME                           VARIANT                        STATUS                         DESCRIPTION
    fraud_training                 quickstart (default)           READY                          Training set for fraud detection.
    fraud_training                 v2                             CREATED                        Improved training set for fraud detection.
    recommender                    v1 (default)                   CREATED                        Training set for recommender system.
    ```

    ``` py title="Input"
    print(training_sets_list)
    ```

    ``` json title="Output"
    // list_training_sets returns a list of TrainingSet objects

    [name: "fraud_training"
    default_variant: "quickstart"
    variants: "quickstart", "v2",
    name: "recommender"
    default_variant: "v1"
    variants: "v1"
    ]
    ```

    Returns:
        training_sets (List[TrainingSet]): List of TrainingSet Objects
    """
    return list_name_variant_status_desc(self._stub, "training-set")

list_users(local=False)

List all users. Prints a list of all users.

Examples:

Input

users_list = rc.list_users()

Output

// list_users prints out formatted information on all users

NAME                           STATUS
featureformer                  NO_STATUS
featureformers_friend          CREATED

Input

print(features_list)

Output

// list_features returns a list of Feature objects

[name: "featureformer"
features {
name: "avg_transactions"
variant: "quickstart"
}
labels {
name: "fraudulent"
variant: "quickstart"
}
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
sources {
name: "transactions"
variant: "kaggle"
}
sources {
name: "average_user_transaction"
variant: "quickstart"
},
name: "featureformers_friend"
features {
name: "user_age"
variant: "production"
}
sources {
name: "user_profiles"
variant: "production"
}
]

Returns:

Name	Type	Description
users	List[User]	List of User Objects

Source code in src/featureform/register.py

def list_users(self, local=False):
    """List all users. Prints a list of all users.

    **Examples:**
    ``` py title="Input"
    users_list = rc.list_users()
    ```

    ``` json title="Output"
    // list_users prints out formatted information on all users

    NAME                           STATUS
    featureformer                  NO_STATUS
    featureformers_friend          CREATED
    ```

    ``` py title="Input"
    print(features_list)
    ```

    ``` json title="Output"
    // list_features returns a list of Feature objects

    [name: "featureformer"
    features {
    name: "avg_transactions"
    variant: "quickstart"
    }
    labels {
    name: "fraudulent"
    variant: "quickstart"
    }
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    sources {
    name: "transactions"
    variant: "kaggle"
    }
    sources {
    name: "average_user_transaction"
    variant: "quickstart"
    },
    name: "featureformers_friend"
    features {
    name: "user_age"
    variant: "production"
    }
    sources {
    name: "user_profiles"
    variant: "production"
    }
    ]
    ```

    Returns:
        users (List[User]): List of User Objects
    """
    return list_name_status(self._stub, "user")

print_feature(name, variant=None, local=False)

Get a feature. Prints out information on feature, and all variants associated with the feature. If variant is included, print information on that specific variant and all resources associated with it.

Examples:

Input

avg_transactions = rc.get_feature("avg_transactions")

Output

// get_feature prints out formatted information on feature

NAME:                          avg_transactions
STATUS:                        NO_STATUS
-----------------------------------------------
VARIANTS:
quickstart                     default
-----------------------------------------------

Input

print(avg_transactions)

Output

// get_feature returns the Feature object

name: "avg_transactions"
default_variant: "quickstart"
variants: "quickstart"

Input

avg_transactions_variant = ff.get_feature("avg_transactions", "quickstart")

Output

// get_feature with variant provided prints out formatted information on feature variant

NAME:                          avg_transactions
VARIANT:                       quickstart
TYPE:                          float32
ENTITY:                        user
OWNER:                         featureformer
PROVIDER:                      redis-quickstart
STATUS:                        NO_STATUS
-----------------------------------------------
SOURCE:
NAME                           VARIANT
average_user_transaction       quickstart
-----------------------------------------------
TRAINING SETS:
NAME                           VARIANT
fraud_training                 quickstart
-----------------------------------------------

Input

print(avg_transactions_variant)

Output

// get_feature returns the FeatureVariant object

name: "avg_transactions"
variant: "quickstart"
source {
name: "average_user_transaction"
variant: "quickstart"
}
type: "float32"
entity: "user"
created {
seconds: 1658168552
nanos: 142461900
}
owner: "featureformer"
provider: "redis-quickstart"
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
columns {
entity: "user_id"
value: "avg_transaction_amt"
}

Parameters:

Name	Type	Description	Default
name	str	Name of feature to be retrieved	required
variant	str	Name of variant of feature	None

Returns:

Name	Type	Description
feature	Union[Feature, FeatureVariant]	Feature or FeatureVariant

Source code in src/featureform/register.py

def print_feature(self, name, variant=None, local=False):
    """Get a feature. Prints out information on feature, and all variants associated with the feature. If variant is included, print information on that specific variant and all resources associated with it.

    **Examples:**

    ``` py title="Input"
    avg_transactions = rc.get_feature("avg_transactions")
    ```

    ``` json title="Output"
    // get_feature prints out formatted information on feature

    NAME:                          avg_transactions
    STATUS:                        NO_STATUS
    -----------------------------------------------
    VARIANTS:
    quickstart                     default
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(avg_transactions)
    ```

    ``` json title="Output"
    // get_feature returns the Feature object

    name: "avg_transactions"
    default_variant: "quickstart"
    variants: "quickstart"
    ```

    ``` py title="Input"
    avg_transactions_variant = ff.get_feature("avg_transactions", "quickstart")
    ```

    ``` json title="Output"
    // get_feature with variant provided prints out formatted information on feature variant

    NAME:                          avg_transactions
    VARIANT:                       quickstart
    TYPE:                          float32
    ENTITY:                        user
    OWNER:                         featureformer
    PROVIDER:                      redis-quickstart
    STATUS:                        NO_STATUS
    -----------------------------------------------
    SOURCE:
    NAME                           VARIANT
    average_user_transaction       quickstart
    -----------------------------------------------
    TRAINING SETS:
    NAME                           VARIANT
    fraud_training                 quickstart
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(avg_transactions_variant)
    ```

    ``` json title="Output"
    // get_feature returns the FeatureVariant object

    name: "avg_transactions"
    variant: "quickstart"
    source {
    name: "average_user_transaction"
    variant: "quickstart"
    }
    type: "float32"
    entity: "user"
    created {
    seconds: 1658168552
    nanos: 142461900
    }
    owner: "featureformer"
    provider: "redis-quickstart"
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    columns {
    entity: "user_id"
    value: "avg_transaction_amt"
    }
    ```

    Args:
        name (str): Name of feature to be retrieved
        variant (str): Name of variant of feature

    Returns:
        feature (Union[Feature, FeatureVariant]): Feature or FeatureVariant
    """
    if not variant:
        return get_resource_info(self._stub, "feature", name)
    return get_feature_variant_info(self._stub, name, variant)

print_label(name, variant=None, local=False)

Get a label. Prints out information on label, and all variants associated with the label. If variant is included, print information on that specific variant and all resources associated with it.

Examples:

Input

fraudulent = rc.get_label("fraudulent")

Output

// get_label prints out formatted information on label

NAME:                          fraudulent
STATUS:                        NO_STATUS
-----------------------------------------------
VARIANTS:
quickstart                     default
-----------------------------------------------

Input

print(fraudulent)

Output

// get_label returns the Label object

name: "fraudulent"
default_variant: "quickstart"
variants: "quickstart"

Input

fraudulent_variant = ff.get_label("fraudulent", "quickstart")

Output

// get_label with variant provided prints out formatted information on label variant

NAME:                          fraudulent
VARIANT:                       quickstart
TYPE:                          bool
ENTITY:                        user
OWNER:                         featureformer
PROVIDER:                      postgres-quickstart
STATUS:                        NO_STATUS
-----------------------------------------------
SOURCE:
NAME                           VARIANT
transactions                   kaggle
-----------------------------------------------
TRAINING SETS:
NAME                           VARIANT
fraud_training                 quickstart
-----------------------------------------------

Input

print(fraudulent_variant)

Output

// get_label returns the LabelVariant object

name: "fraudulent"
variant: "quickstart"
type: "bool"
source {
name: "transactions"
variant: "kaggle"
}
entity: "user"
created {
seconds: 1658168552
nanos: 154924300
}
owner: "featureformer"
provider: "postgres-quickstart"
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
columns {
entity: "customerid"
value: "isfraud"
}

Parameters:

Name	Type	Description	Default
name	str	Name of label to be retrieved	required
variant	str	Name of variant of label	None

Returns:

Name	Type	Description
label	Union[label, LabelVariant]	Label or LabelVariant

Source code in src/featureform/register.py

def print_label(self, name, variant=None, local=False):
    """Get a label. Prints out information on label, and all variants associated with the label. If variant is included, print information on that specific variant and all resources associated with it.

    **Examples:**

    ``` py title="Input"
    fraudulent = rc.get_label("fraudulent")
    ```

    ``` json title="Output"
    // get_label prints out formatted information on label

    NAME:                          fraudulent
    STATUS:                        NO_STATUS
    -----------------------------------------------
    VARIANTS:
    quickstart                     default
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(fraudulent)
    ```

    ``` json title="Output"
    // get_label returns the Label object

    name: "fraudulent"
    default_variant: "quickstart"
    variants: "quickstart"
    ```

    ``` py title="Input"
    fraudulent_variant = ff.get_label("fraudulent", "quickstart")
    ```

    ``` json title="Output"
    // get_label with variant provided prints out formatted information on label variant

    NAME:                          fraudulent
    VARIANT:                       quickstart
    TYPE:                          bool
    ENTITY:                        user
    OWNER:                         featureformer
    PROVIDER:                      postgres-quickstart
    STATUS:                        NO_STATUS
    -----------------------------------------------
    SOURCE:
    NAME                           VARIANT
    transactions                   kaggle
    -----------------------------------------------
    TRAINING SETS:
    NAME                           VARIANT
    fraud_training                 quickstart
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(fraudulent_variant)
    ```

    ``` json title="Output"
    // get_label returns the LabelVariant object

    name: "fraudulent"
    variant: "quickstart"
    type: "bool"
    source {
    name: "transactions"
    variant: "kaggle"
    }
    entity: "user"
    created {
    seconds: 1658168552
    nanos: 154924300
    }
    owner: "featureformer"
    provider: "postgres-quickstart"
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    columns {
    entity: "customerid"
    value: "isfraud"
    }
    ```

    Args:
        name (str): Name of label to be retrieved
        variant (str): Name of variant of label

    Returns:
        label (Union[label, LabelVariant]): Label or LabelVariant
    """
    if not variant:
        return get_resource_info(self._stub, "label", name)
    return get_label_variant_info(self._stub, name, variant)

print_source(name, variant=None, local=False)

Get a source. Prints out information on source, and all variants associated with the source. If variant is included, print information on that specific variant and all resources associated with it.

Examples:

Input

transactions = rc.get_transactions("transactions")

Output

// get_source prints out formatted information on source

NAME:                          transactions
STATUS:                        NO_STATUS
-----------------------------------------------
VARIANTS:
kaggle                         default
-----------------------------------------------

Input

print(transactions)

Output

// get_source returns the Source object

name: "transactions"
default_variant: "kaggle"
variants: "kaggle"

Input

transactions_variant = rc.get_source("transactions", "kaggle")

Output

// get_source with variant provided prints out formatted information on source variant

NAME:                          transactions
VARIANT:                       kaggle
OWNER:                         featureformer
DESCRIPTION:                   Fraud Dataset From Kaggle
PROVIDER:                      postgres-quickstart
STATUS:                        NO_STATUS
-----------------------------------------------
DEFINITION:
TRANSFORMATION

-----------------------------------------------
SOURCES
NAME                           VARIANT
-----------------------------------------------
PRIMARY DATA
Transactions
FEATURES:
NAME                           VARIANT
-----------------------------------------------
LABELS:
NAME                           VARIANT
fraudulent                     quickstart
-----------------------------------------------
TRAINING SETS:
NAME                           VARIANT
fraud_training                 quickstart
-----------------------------------------------

Input

print(transactions_variant)

Output

// get_source returns the SourceVariant object

name: "transactions"
variant: "kaggle"
owner: "featureformer"
description: "Fraud Dataset From Kaggle"
provider: "postgres-quickstart"
created {
seconds: 1658168552
nanos: 128768000
}
trainingsets {
name: "fraud_training"
variant: "quickstart"
}
labels {
name: "fraudulent"
variant: "quickstart"
}
primaryData {
table {
    name: "Transactions"
}
}

Parameters:

Name	Type	Description	Default
name	str	Name of source to be retrieved	required
variant	str	Name of variant of source	None

Returns:

Name	Type	Description
source	Union[Source, SourceVariant]	Source or SourceVariant

Source code in src/featureform/register.py

def print_source(self, name, variant=None, local=False):
    """Get a source. Prints out information on source, and all variants associated with the source. If variant is included, print information on that specific variant and all resources associated with it.

    **Examples:**

    ``` py title="Input"
    transactions = rc.get_transactions("transactions")
    ```

    ``` json title="Output"
    // get_source prints out formatted information on source

    NAME:                          transactions
    STATUS:                        NO_STATUS
    -----------------------------------------------
    VARIANTS:
    kaggle                         default
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(transactions)
    ```

    ``` json title="Output"
    // get_source returns the Source object

    name: "transactions"
    default_variant: "kaggle"
    variants: "kaggle"
    ```

    ``` py title="Input"
    transactions_variant = rc.get_source("transactions", "kaggle")
    ```

    ``` json title="Output"
    // get_source with variant provided prints out formatted information on source variant

    NAME:                          transactions
    VARIANT:                       kaggle
    OWNER:                         featureformer
    DESCRIPTION:                   Fraud Dataset From Kaggle
    PROVIDER:                      postgres-quickstart
    STATUS:                        NO_STATUS
    -----------------------------------------------
    DEFINITION:
    TRANSFORMATION

    -----------------------------------------------
    SOURCES
    NAME                           VARIANT
    -----------------------------------------------
    PRIMARY DATA
    Transactions
    FEATURES:
    NAME                           VARIANT
    -----------------------------------------------
    LABELS:
    NAME                           VARIANT
    fraudulent                     quickstart
    -----------------------------------------------
    TRAINING SETS:
    NAME                           VARIANT
    fraud_training                 quickstart
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(transactions_variant)
    ```

    ``` json title="Output"
    // get_source returns the SourceVariant object

    name: "transactions"
    variant: "kaggle"
    owner: "featureformer"
    description: "Fraud Dataset From Kaggle"
    provider: "postgres-quickstart"
    created {
    seconds: 1658168552
    nanos: 128768000
    }
    trainingsets {
    name: "fraud_training"
    variant: "quickstart"
    }
    labels {
    name: "fraudulent"
    variant: "quickstart"
    }
    primaryData {
    table {
        name: "Transactions"
    }
    }
    ```

    Args:
        name (str): Name of source to be retrieved
        variant (str): Name of variant of source

    Returns:
        source (Union[Source, SourceVariant]): Source or SourceVariant
    """
    if not variant:
        return get_resource_info(self._stub, "source", name)
    return get_source_variant_info(self._stub, name, variant)

print_training_set(name, variant=None, local=False)

Get a training set. Prints out information on training set, and all variants associated with the training set. If variant is included, print information on that specific variant and all resources associated with it.

Examples:

Input

fraud_training = rc.get_training_set("fraud_training")

Output

// get_training_set prints out formatted information on training set

NAME:                          fraud_training
STATUS:                        NO_STATUS
-----------------------------------------------
VARIANTS:
quickstart                     default
-----------------------------------------------

Input

print(fraud_training)

Output

// get_training_set returns the TrainingSet object

name: "fraud_training"
default_variant: "quickstart"
variants: "quickstart"

Input

fraudulent_variant = ff.get_training set("fraudulent", "quickstart")

Output

// get_training_set with variant provided prints out formatted information on training set variant

NAME:                          fraud_training
VARIANT:                       quickstart
OWNER:                         featureformer
PROVIDER:                      postgres-quickstart
STATUS:                        NO_STATUS
-----------------------------------------------
LABEL:
NAME                           VARIANT
fraudulent                     quickstart
-----------------------------------------------
FEATURES:
NAME                           VARIANT
avg_transactions               quickstart
-----------------------------------------------

Input

print(fraudulent_variant)

Output

// get_training_set returns the TrainingSetVariant object

name: "fraud_training"
variant: "quickstart"
owner: "featureformer"
created {
seconds: 1658168552
nanos: 157934800
}
provider: "postgres-quickstart"
features {
name: "avg_transactions"
variant: "quickstart"
}
label {
name: "fraudulent"
variant: "quickstart"
}

Parameters:

Name	Type	Description	Default
name	str	Name of training set to be retrieved	required
variant	str	Name of variant of training set	None

Returns:

Name	Type	Description
training_set	Union[TrainingSet, TrainingSetVariant]	TrainingSet or TrainingSetVariant

Source code in src/featureform/register.py

def print_training_set(self, name, variant=None, local=False):
    """Get a training set. Prints out information on training set, and all variants associated with the training set. If variant is included, print information on that specific variant and all resources associated with it.

    **Examples:**

    ``` py title="Input"
    fraud_training = rc.get_training_set("fraud_training")
    ```

    ``` json title="Output"
    // get_training_set prints out formatted information on training set

    NAME:                          fraud_training
    STATUS:                        NO_STATUS
    -----------------------------------------------
    VARIANTS:
    quickstart                     default
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(fraud_training)
    ```

    ``` json title="Output"
    // get_training_set returns the TrainingSet object

    name: "fraud_training"
    default_variant: "quickstart"
    variants: "quickstart"
    ```

    ``` py title="Input"
    fraudulent_variant = ff.get_training set("fraudulent", "quickstart")
    ```

    ``` json title="Output"
    // get_training_set with variant provided prints out formatted information on training set variant

    NAME:                          fraud_training
    VARIANT:                       quickstart
    OWNER:                         featureformer
    PROVIDER:                      postgres-quickstart
    STATUS:                        NO_STATUS
    -----------------------------------------------
    LABEL:
    NAME                           VARIANT
    fraudulent                     quickstart
    -----------------------------------------------
    FEATURES:
    NAME                           VARIANT
    avg_transactions               quickstart
    -----------------------------------------------
    ```

    ``` py title="Input"
    print(fraudulent_variant)
    ```

    ``` json title="Output"
    // get_training_set returns the TrainingSetVariant object

    name: "fraud_training"
    variant: "quickstart"
    owner: "featureformer"
    created {
    seconds: 1658168552
    nanos: 157934800
    }
    provider: "postgres-quickstart"
    features {
    name: "avg_transactions"
    variant: "quickstart"
    }
    label {
    name: "fraudulent"
    variant: "quickstart"
    }
    ```

    Args:
        name (str): Name of training set to be retrieved
        variant (str): Name of variant of training set

    Returns:
        training_set (Union[TrainingSet, TrainingSetVariant]): TrainingSet or TrainingSetVariant
    """
    if not variant:
        return get_resource_info(self._stub, "training-set", name)
    return get_training_set_variant_info(self._stub, name, variant)

search(raw_query, local=False)

Search for registered resources. Prints a list of results.

Examples:

Input

providers_list = rc.search("transact")

Output

// search prints out formatted information on all matches

NAME                           VARIANT            TYPE
avg_transactions               default            Source

Source code in src/featureform/register.py

def search(self, raw_query, local=False):
    """Search for registered resources. Prints a list of results.

    **Examples:**
    ``` py title="Input"
    providers_list = rc.search("transact")
    ```

    ``` json title="Output"
    // search prints out formatted information on all matches

    NAME                           VARIANT            TYPE
    avg_transactions               default            Source
    ```
    """
    if type(raw_query) != str or len(raw_query) == 0:
        raise Exception("query must be string and cannot be empty")
    processed_query = raw_query.translate({ord(i): None for i in ".,-@!*#"})
    return search(processed_query, self._host)

featureform.register.Registrar

These functions are used to register new resources and retrieving existing resources. Retrieved resources can be used to register additional resources.

definitions.py

import featureform as ff

# e.g. registering a new provider
redis = ff.register_redis(
    name="redis-quickstart",
    host="quickstart-redis",  # The internal dns name for redis
    port=6379,
    description="A Redis deployment we created for the Featureform quickstart"
)

Source code in src/featureform/register.py

1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242