Postgres

Overview

The Skyrun SDK offers versatile data preparation capabilities, allowing users to connect to databases or upload files directly for deep learning model training. This page outlines the steps for setting up data connectors and preparing your data using the Skyrun SDK.

Getting Started

Before you begin, ensure that the Skyrun SDK is installed and properly configured in your development environment. For installation instructions, refer to the Skyrun SDK Installation Guide.

Data Preparation

Data preparation in Skyrun SDK can be performed in two primary ways:

  1. Data Connection: Connect directly to a supported database.

  2. Upload File: Upload a data file from your local system.

Option 1: Data Connection

The SDK supports creating a connection to various databases, including PostgreSQL. Below is a step-by-step guide to setting up a PostgreSQL connection.

Step 1: Create a Data Connector

To connect to a PostgreSQL database, use the skyrun.create_connector method:

skyrun.create_connector(
    connection_name="aa_connection_postgres",
    connection_type="postgres",
    db_host='aria-default-workspace-test5555.crg3mfkamsv8.us-east-2.rds.amazonaws.com',
    db_name="test5555",
    db_username="default_username",
    db_password="default_password",
    db_port=5432
)

Parameters:

  • connection_name: Unique identifier for the connection.

  • connection_type: Type of the database (e.g., "postgres").

  • db_host: Database host address.

  • db_name: Database schema.

  • db_username: Username for the database.

  • db_password: Password for the database.

  • db_port: Database port.

Step 2: Prepare Data

After establishing the connection, prepare your data with skyrun.prepare_data:

skyrun.prepare_data(
    type='connection',
    data_source_name='second',
    connection_name='my_new_connection_postgres',
    table_name='movieinteractions',
    user_column_name='USER_IDS',
    item_column_name='MOVIE_IDS'
)

Parameters:

  • type: Specifies the type of data source ("connection").

  • data_source_name: A name for the data source.

  • connection_name: Name of the connection to use.

  • table_name: Name of the table to prepare data from.

Option 2: Upload File

For uploading a local file, the skyrun.prepare_data method is used:

skyrun.prepare_data(
    type='file',
    data_source_name='first',
    file_path="/home/bs/aaProjects/pigeons-playground/Skyrun-SDK/user_product_purchases.csv"
    user_column_name='USER_IDS',
    item_column_name='MOVIE_IDS'
)

Parameters:

  • type: Specifies the type of data source ("file").

  • data_source_name: A name for the data source.

  • file_path: Path to the file to be uploaded.

For further information or support, please refer to the Skyrun SDK User Guide or contact our support team.