Getting Started with FastTransfer

FastTransfer is a high-performance cross-database data transfer tool that enables fast, parallel data movement between different database systems.

If you haven't installed FastTransfer yet, please check the Installation section first.

You can also build your first FastTransfer command using our interactive Command Builder Wizard.

What is FastTransfer?

FastTransfer is designed to transfer data between databases efficiently:

• Cross-Database: Transfer data between different database systems (PostgreSQL ↔ SQL Server, MySQL → Oracle, etc.)
• High Performance: Parallel processing capabilities for large datasets
• Simple CLI: Easy-to-use command-line interface
• Flexible: Support for multiple connection types and authentication methods
• Production-Ready: Built for enterprise data pipelines and ETL workflows

Supported Databases

As Source

• Microsoft SQL Server
• PostgreSQL
• MySQL / MariaDB
• Oracle
• IBM Netezza
• SAP HANA
• Teradata
• ClickHouse
• DuckDB
• ODBC / OLEDB connections

As Target

• Microsoft SQL Server
• PostgreSQL (with COPY protocol support)
• MySQL / MariaDB
• Oracle (with direct path and bulk insert)
• IBM Netezza
• SAP HANA (with bulk insert)
• Teradata
• ClickHouse
• DuckDB
• ODBC / OLEDB connections

See the Compatibility page for platform-specific support details.

Basic Concepts

Transfer Architecture

A FastTransfer operation consists of:

1. Source Connection: Where data comes from (source database, table, or query)
2. Target Connection: Where data goes to (target database and table)
3. Parallelization (optional): How data is split across multiple threads for faster transfer
4. Mapping: How source columns map to target columns

Prerequisites

Before transferring data, ensure:

• ✅ Source table/query exists and is accessible
• ✅ Target table exists with compatible schema
• ✅ Network connectivity between your machine and both databases
• ✅ Appropriate permissions on both source (SELECT) and target (INSERT)
• ✅ FastTransfer installed and license file available (if required)

Target Table Must Exist

FastTransfer does not create target tables automatically. You must create the target table with appropriate column definitions before running a transfer.

Your First Transfer

Let's start with a simple table transfer from PostgreSQL to SQL Server.

Step 1: Create Target Table

First, create the target table in SQL Server:

-- On SQL Server
CREATE TABLE dbo.customers (
    customer_id INT PRIMARY KEY,
    first_name VARCHAR(100),
    last_name VARCHAR(100),
    email VARCHAR(255),
    created_at DATETIME
);

Step 2: Run the Transfer

Windows

.\FastTransfer.exe `
  --sourceconnectiontype pgsql `
  --sourceserver "pgserver.example.com" `
  --sourceuser "pguser" `
  --sourcepassword "pgpass" `
  --sourcedatabase "sales" `
  --sourceschema "public" `
  --sourcetable "customers" `
  --targetconnectiontype mssql `
  --targetserver "sqlserver.example.com" `
  --targetuser "sqluser" `
  --targetpassword "sqlpass" `
  --targetdatabase "warehouse" `
  --targetschema "dbo" `
  --targettable "customers"

Linux

./FastTransfer \
  --sourceconnectiontype pgsql \
  --sourceserver "pgserver.example.com" \
  --sourceuser "pguser" \
  --sourcepassword "pgpass" \
  --sourcedatabase "sales" \
  --sourceschema "public" \
  --sourcetable "customers" \
  --targetconnectiontype mssql \
  --targetserver "sqlserver.example.com" \
  --targetuser "sqluser" \
  --targetpassword "sqlpass" \
  --targetdatabase "warehouse" \
  --targetschema "dbo" \
  --targettable "customers"

That's it! FastTransfer will:

1. Connect to the PostgreSQL source
2. Connect to the SQL Server target
3. Read data from sales.public.customers
4. Write data to warehouse.dbo.customers
5. Display progress and completion statistics

Common Transfer Scenarios

Scenario 1: Full Table Copy

Transfer an entire table from one database to another:

./FastTransfer \
  --sourceconnectiontype mysql \
  --sourceserver "mysql.example.com:3306" \
  --sourceuser "mysqluser" \
  --sourcepassword "mysqlpass" \
  --sourcedatabase "ecommerce" \
  --sourcetable "orders" \
  --targetconnectiontype pgsql \
  --targetserver "pg.example.com" \
  --targetuser "pguser" \
  --targetpassword "pgpass" \
  --targetdatabase "analytics" \
  --targetschema "public" \
  --targettable "orders"

Scenario 2: Transfer with Query Filter

Transfer only specific rows using a SQL query:

./FastTransfer \
  --sourceconnectiontype mssql \
  --sourceserver "sqlserver.example.com" \
  --sourceuser "sqluser" \
  --sourcepassword "sqlpass" \
  --sourcedatabase "sales" \
  --query "SELECT order_id, customer_id, order_date, total FROM dbo.orders WHERE order_date >= '2024-01-01'" \
  --targetconnectiontype pgsql \
  --targetserver "pg.example.com" \
  --targetuser "pguser" \
  --targetpassword "pgpass" \
  --targetdatabase "analytics" \
  --targetschema "public" \
  --targettable "recent_orders"

Scenario 3: Truncate and Reload

Replace all data in the target table:

./FastTransfer \
  --sourceconnectiontype pgsql \
  --sourceserver "pg.example.com" \
  --sourcedatabase "production" \
  --sourcetable "product_catalog" \
  --targetconnectiontype mssql \
  --targetserver "sqlserver.example.com" \
  --targetdatabase "warehouse" \
  --targettable "products" \
  --loadmode Truncate

Truncate Mode

Using --loadmode Truncate will delete all existing data in the target table before loading new data. Use with caution!

Scenario 4: Parallel Transfer for Large Tables

Use parallel processing for faster transfers of large tables:

./FastTransfer \
  --sourceconnectiontype pgsql \
  --sourceserver "pg.example.com" \
  --sourcedatabase "warehouse" \
  --sourcetable "large_fact_table" \
  --targetconnectiontype mssql \
  --targetserver "sqlserver.example.com" \
  --targetdatabase "analytics" \
  --targettable "fact_table" \
  --method Random \
  --degree 4

This uses 4 parallel threads to transfer data simultaneously. See Parallel Parameters for more details.

Understanding Parameters

Required Parameters

Every transfer needs these core parameters:

Source Connection:

• --sourceconnectiontype - Database type (mssql, pgsql, mysql, oracle, etc.)
• --sourceserver - Server name or IP address
• Authentication: --sourceuser & --sourcepassword OR --sourcetrusted

Source Location:

• --sourcedatabase - Source database name
• --sourcetable - Source table name (OR use --query for custom SQL)

Target Connection:

• --targetconnectiontype - Target database type
• --targetserver - Target server name or IP
• Authentication: --targetuser & --targetpassword OR --targettrusted

Target Location:

• --targetdatabase - Target database name
• --targettable - Target table name

Optional Parameters

Enhance your transfers with these optional parameters:

• --loadmode - Append (default) or Truncate
• --mapmethod - Position (default) or Name mapping
• --batchsize - Rows per batch (default varies by database)
• --method - Parallelization method (None, Random, DataDriven, etc.)
• --degree - Number of parallel threads
• --useworktables - Use staging tables for better performance
• --loglevel - Information (default) or Debug
• --runid - Custom identifier for tracking

Next Steps

Now that you've completed your first transfer, explore these topics:

• CLI Parameters - Detailed reference for all parameters
• Parallel Processing - Speed up large transfers
• Examples - Real-world transfer scenarios
• Logging - Configure logging for monitoring and troubleshooting
• Limitations - Important restrictions to be aware of

Getting Help

If you encounter issues:

1. Enable debug logging: Add --loglevel Debug to see detailed information
2. Check logs: Review the log output for error messages
3. Verify connections: Test source and target connections with native tools (psql, sqlcmd, mysql)
4. Review documentation: Check specific pages for your source and target databases
5. Contact support: Reach out to support@arpe.io

Quick Reference

PostgreSQL → SQL Server

./FastTransfer \
  --sourceconnectiontype pgsql --sourceserver pg.example.com \
  --targetconnectiontype mssql --targetserver sql.example.com \
  --sourcedatabase mydb --sourcetable mytable \
  --targetdatabase mydb --targettable mytable

SQL Server → PostgreSQL

./FastTransfer \
  --sourceconnectiontype mssql --sourceserver sql.example.com \
  --targetconnectiontype pgcopy --targetserver pg.example.com \
  --sourcedatabase mydb --sourcetable mytable \
  --targetdatabase mydb --targettable mytable

MySQL → Oracle

./FastTransfer \
  --sourceconnectiontype mysql --sourceserver mysql.example.com:3306 \
  --targetconnectiontype orabdp --targetserver oracle.example.com:1521/ORCLPDB \
  --sourcedatabase mydb --sourcetable mytable \
  --targettable SCHEMA.MYTABLE

Use the Wizard

Not sure about the exact syntax? Use our interactive wizard to build your command with a visual interface!