Upgrade PostgreSQL from 9.1 to 9.3 on Kubuntu

This seven steps will perform the upgrade of PostgreSQL from version 9.1 to version 9.3.
This also works in Ubuntu and it can also be used to upgrade between any version numbers.

To upgrade between any versions, just changed the 9.1 for the legacy version number and the 9.3 for the new version number.

First install the necessary dependencies

sudo apt-get update
sudo apt-get -y install python-software-properties

Second, add the PostgreSQL repository

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

Third, setup the repository

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" >> /etc/apt/sources.list.d/postgresql.list'

Fourth, install PostgreSQL 9.3.

sudo apt-get install postgresql-9.3 postgresql-server-dev-9.3 postgresql-contrib-9.3

Fifth, perform the upgrade process.
The upgrade process is performed by having both servers running at the same time. Note that the new 9.3 version will run on a different port, 5433 as specified in the script bellow, and it will be set to the default port latter when the legacy version is uninstalled and the default port becomes available.

sudo su -l postgres
psql -d template1 -p 5433
CREATE EXTENSION IF NOT EXISTS hstore;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
\q #logout from database
service postgresql stop
/usr/lib/postgresql/9.3/bin/pg_upgrade -b /usr/lib/postgresql/9.1/bin -B /usr/lib/postgresql/9.3/bin -d /var/lib/postgresql/9.1/main/ -D /var/lib/postgresql/9.3/main/ -O " -c config_file=/etc/postgresql/9.3/main/postgresql.conf" -o "-c config_file=/etc/postgresql/9.1/main/postgresql.conf"
exit # logout postgresql back to previous user

Sixth, remove the 9.1 version.

sudo apt-get remove postgresql-9.1

Seventh, set the new version server port back to the default value and restart the service.

sudo vim /etc/postgresql/9.3/main/postgresql.conf # find old port of 5433 and change it to 5432
sudo service postgresql restart

For XML Path Equivalent in Oracle For String Aggregation

During a migration from SQL Server to Oracle 11g, I’ve came across with a specific functionality of T-SQL FOR XML PATH.
In the scope of the migration, the result of the select clause where this was applied was the concatenation of the values of a column per key.

Here’s an example of what I’ve found on T-SQL during the migration:

Declare @tbl1 table (
  id int,
  Col1 varchar(50),
  Col2 varchar(50)
)

INSERT INTO @tbl1
SELECT 1,'A001','Y'

INSERT INTO @tbl1
SELECT 2,'A002','N'

INSERT INTO @tbl1
SELECT 3,'A003','N'

INSERT INTO @tbl1
SELECT 4,'A004','Y'

INSERT INTO @tbl1
SELECT 5,'A005','N'

INSERT INTO @tbl1
SELECT 6,'A006','N'

INSERT INTO @tbl1
SELECT 7,'A007','N'

INSERT INTO @tbl1
SELECT 8,'A008','Y'

-- Show current values
select * from @tbl1

-- Get all values when col2 has value 'N'
SELECT c.col1 AS [text()]
FROM @tbl1 c
WHERE c.col2 = 'N'
FOR XML PATH(''),type

-- Aggregate all values from col1 using col2 as key
select c1.col2 [Yes/No],
  (SELECT c.col1 AS [text()]
   FROM @tbl1 c
   WHERE c.col2 = c1.col2
   FOR XML PATH(''),type) as Value
FROM @tbl1 c1
group by c1.col2
order by c1.col2

The execution of this T-SQL script shows how it is possible transform the result of a query into a new result where the values are aggregated by col1, and all col2 values are concatenated.

In order to achieve this kind of aggregate transformation in Oracle, one has to make use of an associative array, as Tom explains in the “String Concatenation” how one can do this. Using the examples in the AskTom web site, first we create the “STRING_AGG_TYPE” type:

CREATE OR REPLACE TYPE string_agg_type AS object
(
  total VARCHAR2(4000),

  static FUNCTION ODCIAggregateInitialize(sctx IN OUT string_agg_type) RETURN NUMBER,

  member FUNCTION ODCIAggregateIterate(self IN OUT string_agg_type, VALUE IN VARCHAR2) RETURN NUMBER,

  member FUNCTION ODCIAggregateTerminate(self IN string_agg_type, returnValue OUT VARCHAR2, flags IN NUMBER) RETURN NUMBER,

  member FUNCTION ODCIAggregateMerge(self IN OUT string_agg_type, ctx2 IN string_agg_type) RETURN NUMBER
);
/

And the correspondent body:

CREATE OR REPLACE TYPE BODY string_agg_type
IS

static FUNCTION ODCIAggregateInitialize(sctx IN OUT string_agg_type)
RETURN NUMBER
IS
BEGIN
  sctx := string_agg_type( NULL );
  RETURN ODCIConst.Success;
END;

member FUNCTION ODCIAggregateIterate(self IN OUT string_agg_type, VALUE IN VARCHAR2 )
RETURN NUMBER
IS
BEGIN
  -- NOTE: Insert a separator in the empty string if you wish
  self.total := self.total || '' || VALUE;
  RETURN ODCIConst.Success;
END;

member FUNCTION ODCIAggregateTerminate(self IN string_agg_type, returnValue OUT VARCHAR2, flags IN NUMBER)
RETURN NUMBER
IS
BEGIN
  -- NOTE: ltrim(self.total, '') if you've specified a separator in ODCIAggregateIterate
  returnValue := LTRIM(self.total);
  RETURN ODCIConst.Success;
END;

member FUNCTION ODCIAggregateMerge(self IN OUT string_agg_type, ctx2 IN string_agg_type)
RETURN NUMBER
IS
BEGIN
  self.total := self.total || ctx2.total;
  RETURN ODCIConst.Success;
END;

END;
/

Then we create the aggregated function “stragg” of the type we’ve just created:

CREATE OR REPLACE FUNCTION stragg(input VARCHAR2 ) RETURN VARCHAR2 PARALLEL_ENABLE AGGREGATE USING string_agg_type;
/

We are now able to make a similar aggregation in Oracle:

CREATE TABLE tbl1 (
  id NUMBER(10),
  Col1 VARCHAR(50),
  Col2 VARCHAR(50)
);
/

INSERT INTO tbl1 (id, col1, col2)
VALUES (1,'A001','Y');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (2,'A002','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (3,'A003','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (4,'A004','Y');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (5,'A005','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (6,'A006','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (7,'A007','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (8,'A008','Y');
/
COMMIT;
/

-- Show current values
SELECT * FROM tbl1;
/

-- Get all values when col2 has value 'N'
SELECT c.col2, STRAGG(c.col1)
FROM tbl1 c
WHERE c.col2 = 'N'
GROUP BY c.col2;
/

-- Aggregate all values from col1 using col2 as key
SELECT c1.col2 "Yes/No",
  (SELECT STRAGG(c.col1)
   FROM tbl1 c
   WHERE c.col2 = c1.col2
  ) AS val
FROM tbl1 c1
GROUP BY c1.col2
ORDER BY c1.col2;
/

-- drop table tbl1;

And that’s it. This will work in 9i and beyond, for 8i, check the 8i workaround in the same AskTom thread.

Interpreting DB2 JDBC error messages

On data migration projects using BD2, sometime there are some awkward DB2/JDBC errors.
It does not matter if it is DB2 on Windows, Unix or iSeries AS/400.

Part of the problem is that the error messages come in localized, on our case in Portuguese, making the debug task a lot harder since it’s almost impossible to find decent help by searching through the error messages. The other part of the problem is that the SQL error codes sometimes are unclear.
Here’s an example of a common problem that we have in data migration projects:

com.ibm.db2.jcc.b.rg: [jcc][t4][102][10040][3.50.152] Non-atomic batch failure. The batch was submitted, but at least one exception occurred
on an individual member of the batch.
Use getNextException() to retrieve the exceptions for specific batched elements.

com.ibm.db2.jcc.b.pm: Error for batch element #1: The current
transaction was rolled back because of error "-289".. SQLCODE=-1476,
SQLSTATE=40506, DRIVER=3.50.152

com.ibm.db2.jcc.b.SqlException: [jcc][103][10843][3.50.152]
[...] ERRORCODE=-4225, SQLSTATE=null

Following DB2 official documentation:

  • SQLCODE=-1476 means that the current transaction was rolled back because of one of the listed errors, but none of them made much sense since none of the errors identified seem to apply.
  • SQLSTATE=40506 means that the current transaction was rolled back because of an SQL error, which is basically the same as the SQLCODE above.
  • ERRORCODE=-4225 means an error occurred when data was sent to a server or received from a server, which is totally useless.

The real useful information is hidden in the second error message, in the ‘transaction was rolled back because of error “-289”‘ message. The key here is the -289 error.
This error means “Unable to allocate new pages in table space“, and this is the real cause for such a big fuss.

In data migration projects it’s common some of the DB2 table spaces to run out of space. But with such an error message, the error itself is kind of hidden in the middle of the stack trace, all that is shown is a loose error code…
For the sake of the developers productivity, the cause of the error should be highlighted and perfectly visible and understood in order to be quickly identified and fixed.