// List all databases using \l+ with more details (including description, tablespace & DB size) (psql)
Help on CREATE DATABASE command syntax
\h CREATE DATABASE
// Help on SQL Command Syntax (For e.g. CREATE DATABASE) (psql)
Create database
CREATE DATABASE mytest;
// Creates a new database “mytest” (SQL)
By default, the owner of the database is the current login user.
\c test
You are now connected to database "test" as user "postgres".
// Connect to a PostgreSQL database “test” as “postgres” user (psql)
Tables
Show table
\d TABLE_NAME
// Show table definition including indexes, constraints & triggers (psql)
Show details
\d+ TABLE_NAME
// More detailed table definition including description and physical disk size (psql)
List tables from current schema
\dt
// List tables from current schema (psql)
List tables from all schemas
\dt *.*
// List tables from all schemas (psql)
List tables for a schema
\dt <name-of-schema>.*
// List the tables in a specific schema (psql)
Copy table data to CSV file
\copy (SELECT * FROM __table_name__) TO 'file_path_and_name.csv' WITH CSV
// Export a table as CSV (psql)
Check indexes for a table using sql
SELECT * FROM pg_indexes WHERE tablename='__table_name__' AND
schemaname='__schema_name__';
// Show table indexes (SQL)
Collects statistics about the contents of tables
ANALYZE [__table__]
// Analyze a table and store the results in the pg_statistic system catalog (SQL)
With no parameter, ANALYZE examines every table in the current database
Adding comment on table/column
Comment on table employee is 'Stores employee records';
// Comment on table (SQL)
Comment on column employee.ssn is 'Employee Social Security Number';
// Comment on column (SQL)
Approximate table row count / table cardinality
SELECT reltuples AS card FROM pg_class WHERE relname = '<table_name>';
// Use this to do fast (but not exact) counts from tables. Helpful if a table has millions/billions of records and you just want estimated rows quickly. (SQL)
postgres@localhost:~$ psql
psql (9.6.12)Type "help" for help.postgres=#
// Enter PostgreSQL Command Line via “psql” client (psql)
Connect database
\c test
You are now connected to database "test" as user "postgres".
// Connect to a PostgreSQL database “test” as “postgres” user (psql)
Check psql client version
$ psql -V
psql (PostgreSQL) 9.6.12
// Check psql client version (psql)
Check Postgres server version
select version();
version
-----------------------------------------------------------------------------------------------------------
PostgreSQL 9.6.12 on x86_64-pc-linux-gnu, compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit
(1 row)
// Check Postgres server version (SQL)
Queries
Create a new table
CREATE TABLE IF NOT EXISTS employee (
emp_id SERIAL PRIMARY KEY, -- AUTO_INCREMENT integer, as primary key
emp_name VARCHAR(50) NOT NULL,
emp_salary NUMERIC(9,2) NOT NULL
);
// Creates a new table (SQL)
Display table
\d employee
Table "public.employee"
Column | Type | Modifiers
------------+-----------------------+-----------------------------------------------------------
emp_id | integer | not null default nextval('employee_emp_id_seq'::regclass)
emp_name | character varying(50) | not null
emp_salar | numeric(9,2) | not null
Indexes:
"employee_pkey" PRIMARY KEY, btree (emp_id)
// Display table (psql)
select * from employee where emp_salary >= 5000;
emp_id | emp_name | emp_salary
--------+----------+------------
1 | John | 5000.00
3 | Robert | 7500.50
(2 rows)
// Select data based on filter condition (e.g. emp_salary >= 5000) (SQL)
Conditional update query (Safe Update);
BEGIN;
update employee set emp_salary = 6000 where emp_name = 'John';
COMMIT;
// Update record based on a condition (e.g. Update emp_salary for employee ‘John’) (SQL)
Records are not committed inside database unless you issue a commit. Updates can be undone as well if you issue ROLLBACK command instead of COMMIT.
Functions
Create a new function
$$
CREATE FUNCTION add(integer, integer) RETURNS integer
AS 'select $1 + $2;'
LANGUAGE SQL
IMMUTABLE
RETURNS NULL ON NULL INPUT;
$$
// Create a function to add 2 integers (SQL)
This function takes two integers as parameters IMMUTABLE means that the function cannot modify the database and always returns the same result when given the same argument values.
Calling function
select add(5,9);
add
-----
14
(1 row)
// Function call (SQL)
List functions
\df
List of functions
Schema | Name | Result data type | Argument data types | Type
--------+------+------------------+---------------------+--------
public | add | integer | integer, integer | normal
(1 row)
// Display all Functions (psql)
\df+
// Display all functions and additional information, including owner, source code, and description, etc. (psql)
Edit function
\ef myfunction
// Edit a function in default editor (psql)
Views
Listing PostgreSQL views helps in comprehending the database structure, optimizing queries, analyzing dependencies, documentation, and maintaining security and access control.
List views
\dv
// List views from current schema (psql)
\dv *.*
// List views from all schemas (psql)
Users
Set/reset Postgres user password
\password username
// To set/reset a password for PostgreSQL database user (psql)
For example: Change password for current user “postgres”:
\password postgres
Enter new password: xxxx
Enter it again: xxxx
Show all users
select * from pg_user;
// Display PostgreSQL database users (SQL)
\du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------------------+-----------
testrole | | {}
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
// Display PostgreSQL database roles (psql)
Login & enter PostgreSQL terminal
$ psql -U testuser mytest
Password for user testuser: ......
psql (9.6.12)
Type "help" for help.
// Login to PostgreSQL: psql -U user database (psql)
Indexes
Create a new index on a table
create index idx_employee_emp_name on employee using btree (emp_name asc);
// Create a new index on the emp_name column of the employee table (SQL)
This index specifies “btree” as the index method and uses “asc” to store the index key column data in ascending order.
View indexes of a table
\d employee
postgres=# \d employee;
Table "public.employee"
Column | Type | Modifiers
------------+-----------------------+-----------------------------------------------------------
emp_id | integer | not null default nextval('employee_emp_id_seq'::regclass)
emp_name | character varying(50) | not null
emp_salary | numeric(9,2) | not null
Indexes:
"employee_pkey" PRIMARY KEY, btree (emp_id)
"idx_employee_emp_name" btree (emp_name)
// List indexes of a table along with table definition (psql)
List all indexes
\di
List of relations
Schema | Name | Type | Owner | Table
--------+-----------------------+-------+----------+----------
public | employee_pkey | index | postgres | employee
public | idx_employee_emp_name | index | postgres | employee
(2 rows)
// List all indexes from all tables (psql)
Database indexes in Timescale work the same as in regular PostgreSQL. When working with Timescale’s hypertables (which abstract partitioning, taking care of it automatically), Timescale will also create the indexes automatically. Try Timescale for free today.
Drop index from a table
drop index idx_employee_emp_name;
// Drop an existing index from a table (SQL)
Constraints
Postgres constraints are rules enforced on data columns within a table to maintain data integrity and prevent the insertion of invalid data.
Create a table with primary & unique constraints
$$
CREATE TABLE IF NOT EXISTS employee (
emp_id SERIAL `PRIMARY` KEY,
emp_name VARCHAR(50) NOT NULL,
emp_ssn VARCHAR (30) NOT NULL `UNIQUE`,
emp_salary NUMERIC(9,2) NOT NULL
);
$$
// Creates a new table with primary & unique key constraints (SQL)
Primary Key Constraint: Enforces the uniqueness of a column or a set of columns, ensuring that each row in a table is uniquely identified.
Unique Constraint: Ensures that all values in a column or a set of columns are distinct, except for null values.
// Insert records in a table with unique key constraints specified (SQL)
This table uses emp_id as primary key column (Keyword “primary”) and a unique constraint (Keyword “unique”) is specified on employee social security number (emp_ssn) to avoid duplicate ssn being entered.
// Insert records in table with check constraints specified (SQL)
Check constraint (Keyword “check”) is specified on order amount (ord_amount > 0) on table so any records with ord_amount <=0 will fail to insert.
Check Constraint: Verifies that all values in a column or a set of columns satisfy a specified condition or expression.
Define relation between two tables (foreign key constraint)
$$
CREATE TABLE IF NOT EXISTS department (
dept_id SERIAL PRIMARY KEY,
dept_name VARCHAR(50) NOT NULL
);
$$
CREATE TABLE
$$
CREATE TABLE IF NOT EXISTS employee (
emp_id SERIAL PRIMARY KEY,
emp_name VARCHAR(50) NOT NULL,
emp_ssn VARCHAR (30) NOT NULL UNIQUE,
emp_salary NUMERIC(9,2) NOT NULL,
emp_dept_id INTEGER `REFERENCES` department (dept_id) -- Foreign Key
);
$$
CREATE TABLE
// Creates table department & employee and defines a relation of an employee to department using foreign key (“REFERENCES”) (SQL)
Foreign Key Constraint: Establishes a link between data in two tables, enforcing referential integrity by preserving the relationships between the linked tables.
// Display constraints on a table (using \d option) (psql)
Identifiers
String concatenate operator [ string || string ]
select 'Gordon' || ' ' || 'Moore' As fullName;
fullname
--------------
Gordon Moore
(1 row)
// Concatenates two or more strings using “||”. (SQL)
This operator can be applied on table columns as well. e.g. “select first_name || ‘ ‘ || last_name As fullName from person”.
Square & cube root operator (|/ & ||/)
select |/25 As sqrt;
sqrt
------
5
(1 row)
// Square root operator. (SQL)
select ||/125 As cubert;
cubert
------
5
(1 row)
// Cube root operator. (SQL)
Factorial operator (!)
select 5! As factorial;
factorial
-----------
120
(1 row)
// Factorial operator. (SQL)
Binary complement operator (~)
select ~60 As compl;
compl
----------
-61
(1 row)
// Binary 2’s complement. This operator has a flipping effect on bits. (SQL)
Assume if A = 60, now in binary format they will be as follows − A = 0011 1100 ~A = 1100 0011 (flipping bits. change 0 to 1 & 1 to 0).
String lower & upper function [ lower(string), upper(string) ]
select lower('Rohit Kumawat') As lowerCase, upper('Rohit Kumawat') As upperCase;
lowercase | uppercase
---------------+---------------
rohit kumawat | ROHIT KUMAWAT
(1 row)
// Postgres lower & upper function (SQL)
Number of characters in string [ char_length(string) ]
select char_length('Arizona') as num_chars;
num_chars
-----------
7
(1 row)
// Number of characters in string (SQL)
Location of specified substring [ position(substring in string) ]
select position('pan' in 'japan') As pos;
pos
-----
3
(1 row)
// Location of specified substring (SQL)
Extract substring [ substring(string from [int] for [int] ]
select substring('postgres' from 3 for 3) As sub_string;
sub_string
------------
stg
(1 row)
// Extract substring ‘postgres’ starting from third character up to three characters length (SQL)
Insert newline in SQL output
postgres=# select 'line 1'||E'\n'||'line 2' As newline;
newline
---------
line 1 +
line 2
(1 row)
// Insert new line using E’\n’ (SQL)
Another option is to use the chr() function. (E is used for the Escape String constant). Here’s a few other escape sequences: \b backspace \f form feed \n newline \r carriage return \t tab.
Quote identifier
Update "my table" set "a&b" = 0;
// Using double quotes as delimited identifier
This allows constructing table or column names that would otherwise not be possible, such as ones containing spaces or ampersands. Also double quotes are used for escaping reserved keywords in PostgreSQL.
Dollar-quoted string constant
select $$Maria’s dogs$$ As col;
col
--------------------
Maria’s dogs
(1 row)
// Use dollar quotes string constant instead of double quotes for strings (SQL)
If a string contains many single quotes or backslashes then PostgreSQL has an alternative called “dollar quoting.”
Maintenance
Garbage collect (reclaim storage)
VACUUM [__Table__]
vacuum(verbose, analyze) employee;
INFO: vacuuming "public.employee"
INFO: scanned index "employee_pkey" to remove 1 row versions
. . .
sample, 1 estimated total rows
VACUUM
// Use the vacuum command to reclaim storage from deleted rows in the employee table (SQL)
1. Table rows that are deleted or obsoleted by an update are not physically removed from their table; they remain present until a VACUUM command is executed. Therefore it’s necessary to do VACUUM periodically, especially on frequently-updated tables.
2. Verbose Prints a detailed vacuum activity report for each table.
3. Analyze update statistics for table.
Gather statistics
ANALYZE [__table__]
analyze verbose employee;
INFO: analyzing "public.employee"
INFO: "employee": scanned 1 of 1 pages, containing 1 live rows and 0 dead rows; 1 rows in sample, 1 estimated total rows
ANALYZE
// Analyzes a table and stores the results in the pg_statistic system catalog (SQL)
1. ANALYZE gathers statistics for the query planner to create the most efficient query execution plans. Accurate statistics assist planner to choose the most appropriate query plan, and thereby improve the speed of query processing.
2. Verbose prints a detailed analyze activity report for each table.
3. With no table name specified, ANALYZE examines every table in the current database.
State - State of Session (e.g., active, waiting, idle ..)
Query - Query executed
Cancel running query
SELECT pg_cancel_backend(pid);
// To cancel a running query with pid provided. This is useful in case of killing long-running queries (SQL)
Biggest PostgreSQL table/indexes by their sizes
SELECT
nspname || '.' || relname AS "Object Name", relkind As "Object Type",
pg_size_pretty(pg_relation_size(C.oid)) AS "size"
FROM pg_class C
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
WHERE nspname NOT IN ('pg_catalog', 'information_schema')
ORDER BY pg_relation_size(C.oid) DESC
LIMIT 20;
// Top 20 big tables/indexes (excluding catalog tables) (SQL)
Backup
Database backup (with default options)
$ pg_dump mydb > mydb.bak.sql
// Create a backup for a database “mydb” in plain-text SQL Script file (mydb.bak.sql) (pg_dump)
// Stops PostgreSQL service through root user (Linux)
service postgresql start
// Starts PostgreSQL service through root user (Linux)
service postgresql restart
// Restarts PostgreSQL service through root user (Linux)
If running from non root user you must prefix your command with “sudo” and non-root user should already be there in sudoer’s list. Also be careful with running these commands because some distributors don’t provide them these days.
Display configuration parameters
show all
// List all current runtime configuration parameters (psql)
Display configuration parameters using sql
select * from pg_settings;
// List all current runtime configuration parameters using SQL with additional details, including description (SQL)
// Display current value set for “max_connections” parameter (SQL)
Show PostgreSQL config file location
show config_file;
config_file
------------------------------------------
/etc/postgresql/9.6/main/postgresql.conf
(1 row)
// Show PostgreSQL configuration file location (psql)
The PostgreSQL configuration files are stored in directory from above command output. The main configuration file is called “postgresql.conf." Want an easier way to find it?
Display contents of Postgres config file location
postgres@localhost:~$ less /etc/postgresql/9.6/main/postgresql.conf
. . . .
data_directory = '/var/lib/postgresql/9.6/main' # use data in another directory
# (change requires restart)
hba_file = '/etc/postgresql/9.6/main/pg_hba.conf' # host-based authentication file
# (change requires restart)
ident_file = '/etc/postgresql/9.6/main/pg_ident.conf' # ident configuration file
# (change requires restart)
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5432
# (change requires restart)
. . . .
(Linux)
– data_directory directive tells where the database files are stored.
– hba_file directive tells the host based authentication file.
– port directive tells the TCP port number. The default is 5432.
Scale PostgreSQL via Partitioning: A Dev’s Intro to Hypertables
Boosting Postgres INSERT Performance by 2x With UNNEST