PostgreSQL Examples
Comprehensive examples for exporting data from PostgreSQL using native connectors (pgsql, pgcopy).
Connection Types
PostgreSQL exports support two connection types:
- pgsql: Standard PostgreSQL protocol
- pgcopy: PostgreSQL COPY protocol (fastest for bulk exports)
Single-threaded Export
Simple export without parallelization.
- Standard Protocol
- COPY Protocol
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--query "SELECT * FROM lineitem LIMIT 1000" `
--directory "C:\temp" `
--fileoutput "pg_lineitem.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "None"
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--query "SELECT * FROM lineitem LIMIT 1000" `
--directory "C:\temp" `
--fileoutput "pg_lineitem.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "None"
Parallel Export - Random Method
Export using Random distribution with modulo operation.
- Standard Protocol
- COPY Protocol
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--query "SELECT * FROM lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_random.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Random" `
--distributekeycolumn "l_linenumber" `
--paralleldegree 8 `
--merge "False" `
--runid "pgsql_to_csv_parallel8_random"
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--query "SELECT * FROM lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_random.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Random" `
--distributekeycolumn "l_linenumber" `
--paralleldegree 8 `
--merge "False" `
--runid "pgcopy_to_csv_parallel8_random"
Parallel Export - DataDriven Method
Export using DataDriven distribution based on distinct column values.
- Standard Protocol
- COPY Protocol
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_dd.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "DataDriven" `
--distributekeycolumn "l_shipmode" `
--paralleldegree 10 `
--merge "False" `
--runid "pgsql_to_csv_parallel10_datadriven"
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_dd.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "DataDriven" `
--distributekeycolumn "l_shipmode" `
--paralleldegree 10 `
--merge "False" `
--runid "pgcopy_to_csv_parallel10_datadriven"
Parallel Export - Ntile Method
Export using Ntile for evenly distributed chunks.
- Standard Protocol
- COPY Protocol
Full table:
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_ntile.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Ntile" `
--distributekeycolumn "l_orderkey" `
--paralleldegree 12 `
--merge "False" `
--runid "pgsql_to_csv_parallel12_ntile"
With query filter:
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--query "SELECT * FROM lineitem WHERE l_shipdate BETWEEN '1995-01-01' AND '1995-12-31'" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_ntile.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Ntile" `
--distributekeycolumn "l_orderkey" `
--paralleldegree 12 `
--merge "False" `
--runid "pgsql_to_csv_parallel12_ntile_filtered"
Full table:
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_ntile.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Ntile" `
--distributekeycolumn "l_orderkey" `
--paralleldegree 12 `
--merge "False" `
--runid "pgcopy_to_csv_parallel12_ntile"
With query filter:
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--query "SELECT * FROM lineitem WHERE l_shipdate BETWEEN '1995-01-01' AND '1995-12-31'" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_ntile.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Ntile" `
--distributekeycolumn "l_orderkey" `
--paralleldegree 12 `
--merge "False" `
--runid "pgcopy_to_csv_parallel12_ntile_filtered"
Parallel Export - Ctid Method
PostgreSQL-specific parallel export using physical row identifier (ctid).
The Ctid method is unique to PostgreSQL and uses the physical row identifier for parallel data distribution. This is very efficient for full table exports.
- Standard Protocol
- COPY Protocol
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_ctid.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Ctid" `
--paralleldegree 8 `
--merge "False" `
--runid "pgsql_to_csv_parallel8_ctid"
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem_ctid.csv" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--parallelmethod "Ctid" `
--paralleldegree 8 `
--merge "False" `
--runid "pgcopy_to_csv_parallel8_ctid"
Export to Parquet
Export PostgreSQL data to Parquet format.
- Standard Protocol
- COPY Protocol
Single-threaded:
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem.parquet" `
--parallelmethod "None"
Parallel with Ctid:
.\\FastTransfer.exe `
--connectiontype "pgsql" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem.parquet" `
--parallelmethod "Ctid" `
--paralleldegree 10 `
--merge "False"
Single-threaded:
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem.parquet" `
--parallelmethod "None"
Parallel with Ctid:
.\\FastTransfer.exe `
--connectiontype "pgcopy" `
--server "localhost" `
--port "5432" `
--database "postgres" `
--user "postgres" `
--password "postgres" `
--sourceschema "public" `
--sourcetable "lineitem" `
--directory "C:\temp" `
--fileoutput "pg_lineitem.parquet" `
--parallelmethod "Ctid" `
--paralleldegree 10 `
--merge "False"
ODBC Connection
Using ODBC DSN for PostgreSQL connection.
Single-threaded:
.\\FastTransfer.exe `
--connectiontype "odbc" `
--dsn "PostgreSQL_DSN" `
--directory "C:\temp" `
--fileoutput "pg_test.csv" `
--query "SELECT * FROM public.test_table" `
--parallelmethod "None" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8"
Parallel Random:
.\\FastTransfer.exe `
--connectiontype "odbc" `
--dsn "PostgreSQL_DSN" `
--directory "C:\temp" `
--fileoutput "pg_test.csv" `
--query "SELECT * FROM public.test_table" `
--parallelmethod "Random" `
--distributekeycolumn "id" `
--paralleldegree 6 `
--merge "False" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8"
Parallel DataDriven:
.\\FastTransfer.exe `
--connectiontype "odbc" `
--dsn "PostgreSQL_DSN" `
--directory "C:\temp" `
--fileoutput "pg_test.csv" `
--query "SELECT * FROM public.test_table" `
--parallelmethod "DataDriven" `
--distributekeycolumn "category" `
--paralleldegree 6 `
--merge "False" `
--decimalseparator "." `
--delimiter "|" `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8"
Build your command with the Wizard