Docker

Minimal, production-ready container image to run FastTransfer, a high-performance data transfer utility designed for data integration and automation workflows.

Image Overview

• Base image: debian:trixie-slim
• Entrypoint: /usr/local/bin/FastTransfer
• Repository: arpe-io/FastTransfer-Image
• DockerHub: arpeio/fasttransfer
• Updates: Published automatically via GitHub Actions for each new release and weekly security updates

info

This setup targets FastTransfer ≥ 0.14.0, which supports passing the license inline via --license "<content>".

Prerequisites

• Docker 24+ (or Podman)
• Valid FastTransfer license (request a trial license)
• Optional: FastTransfer_Settings.json for custom logging configuration

Using the Prebuilt Image from DockerHub

Available Tags

• Version-specific tags: Aligned with FastTransfer releases (e.g., v0.15.0)
• latest tag: Always points to the most recent FastTransfer version

Automatic Updates

• New releases: Images are automatically built when new FastTransfer versions are released
• Security updates: The latest version of each minor branch is automatically rebuilt weekly (every Monday) with the latest base image and security patches

Pull the Image

# Latest version
docker pull arpeio/fasttransfer:latest

# Specific version
docker pull arpeio/fasttransfer:v0.15.0

Basic Commands

# Get command line help
docker run --rm arpeio/fasttransfer:latest -?

# Check version
docker run --rm arpeio/fasttransfer:latest --version

License Requirement

Since version 0.14.0, pass the license content directly via --license "…":

export licenseContent=$(cat ./FastTransfer.lic)

# Use $licenseContent in your docker run commands
docker run --rm arpeio/fasttransfer:latest \
  --license "$licenseContent" \
  [other parameters...]

Best Practice

Prefer --env-file, Docker/Compose/Kubernetes secrets, or managed identities for cloud credentials. Avoid leaving the license content in shell history.

Examples

1. SQL Server → SQL Server (Parallel DB-to-DB Transfer)

export licenseContent=$(cat ./FastTransfer.lic)

docker run --rm \
  arpeio/fasttransfer:latest \
  --sourceconnectiontype "mssql" \
  --sourceserver "host.docker.internal,1433" \
  --sourceuser "SrcUser" \
  --sourcepassword "SrcPass" \
  --sourcedatabase "source_db" \
  --targetconnectiontype "msbulk" \
  --targetserver "host.docker.internal,1433" \
  --targetuser "DestUser" \
  --targetpassword "DestPass" \
  --targetdatabase "dest_db" \
  --method "Ntile" \
  --distributekeycolumn "id" \
  --loadmode "Truncate" \
  --paralleldegree 8 \
  --license "$licenseContent"

2. Import CSV Files to SQL Server in Parallel

export licenseContent=$(cat ./FastTransfer.lic)

docker run --rm \
  -v /local/data:/data \
  arpeio/fasttransfer:latest \
  --sourceconnectiontype "duckdbstream" \
  --sourceserver ":memory:" \
  --query "SELECT * FROM read_csv('/data/lineitem*.CSV')" \
  --targetconnectiontype "msbulk" \
  --targetserver "localhost" \
  --targettrusted \
  --targetdatabase "tpch_test" \
  --targetschema "dbo" \
  --targettable "lineitem" \
  --parallelmethod "Random" \
  --distributeKeyColumn "l_orderkey" \
  --mapmethod "Name" \
  --loadmode "Truncate" \
  --degree 16 \
  --license "$licenseContent"

Configuration

Volumes

The Docker image declares several volumes to organize data and configuration:

VOLUME ["/config", "/data", "/work", "/logs"]

Volume	Purpose	Access Mode	Notes
/config	User-provided configuration files (e.g., Serilog settings)	Read-only / Read-many	Shared across multiple containers; not modified
/data	Input/output data directory for file-based operations	Read-many/Write-many	Stores imported or exported data files
/work	Temporary working directory (container WORKDIR)	Read-many/Write-many	Used internally for temporary processing
/logs	Log output directory (per-run or aggregated logs)	Read-many/Write-many	Stores runtime and execution logs

Configuring FastTransfer Logging

Available from version 0.14.0

FastTransfer supports custom logging configuration through an external Serilog settings file in JSON format.

Example: Logging to Console, Airflow, and Dynamic Log Files

{
  "Serilog": {
    "Using": [
      "Serilog.Sinks.Console",
      "Serilog.Sinks.File",
      "Serilog.Enrichers.Environment",
      "Serilog.Enrichers.Thread",
      "Serilog.Enrichers.Process",
      "Serilog.Enrichers.Context",
      "Serilog.Formatting.Compact"
    ],
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTemplate": "{Timestamp:yyyy-MM-ddTHH:mm:ss.fff zzz} -|- {Application} -|- {runid} -|- {Level:u12} -|- {fulltargetname} -|- {Message}{NewLine}{Exception}",
          "theme": "Serilog.Sinks.SystemConsole.Themes.ConsoleTheme::None, Serilog.Sinks.Console",
          "applyThemeToRedirectedOutput": false
        }
      },
      {
        "Name": "File",
        "Args": {
          "path": "/airflow/xcom/return.json",
          "formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
        }
      },
      {
        "Name": "Map",
        "Args": {
          "to": [
            {
              "Name": "File",
              "Args": {
                "path": "/logs/{logdate}/{sourcedatabase}/log-{filename}-{LogTimestamp}-{TraceId}.json",
                "formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact",
                "rollingInterval": "Infinite",
                "shared": false,
                "encoding": "utf-8"
              }
            }
          ]
        }
      }
    ],
    "Enrich": [
      "FromLogContext",
      "WithMachineName",
      "WithProcessId",
      "WithThreadId"
    ],
    "Properties": {
      "Application": "FastTransfer"
    }
  }
}

note

• If a target directory (such as /logs or /airflow/xcom) does not exist, FastTransfer automatically creates it.
• The file /airflow/xcom/return.json is designed to provide run summaries compatible with Airflow's XCom mechanism.

Available Tokens for Path or Filename Formatting

Token	Description
{logdate}	Current date in yyyy-MM-dd format
{logtimestamp}	Full timestamp of the log entry
{sourcedatabase}	Name of the source database
{sourceschema}	Name of the source schema
{sourcetable}	Name of the source table
{filename}	Name of the file being processed
{runid}	Run identifier provided in the command line
{traceid}	Unique trace identifier generated at runtime

Mounting a Custom Settings File

# First, copy your config file to a volume location
cp ~/FastTransfer_Settings.json /volumes/fasttransfer-config/

# Then run FastTransfer with mounted volumes
docker run --rm \
  -v fasttransfer-config:/config \
  -v fasttransfer-logs:/logs \
  arpeio/fasttransfer:latest \
  --settingsfile "/config/FastTransfer_Settings.json" \
  --sourceconnectiontype "mssql" \
  --sourceserver "host.docker.internal,1433" \
  --sourceuser "SrcUser" \
  --sourcepassword "SrcPass" \
  --sourcedatabase "source_db" \
  --sourceschema "dbo" \
  --sourcetable "orders" \
  --targetconnectiontype "pgcopy" \
  --targetserver "host.docker.internal:5432" \
  --targetuser "PgUser" \
  --targetpassword "PgPass" \
  --targetdatabase "pg_db" \
  --targetschema "dbo" \
  --targettable "orders" \
  --paralleldegree 12 \
  --parallelmethod "Ntile" \
  --distributekeycolumn "o_orderkey" \
  --license "$licenseContent"