Debugging email on Django application development

 Django, Linux, Python, Software Development, Web  Comments Off on Debugging email on Django application development
May 212014
 

One of the common functionalities in Django applications is to send emails, such as for user password resets.

Python SMTP

Since Python comes with it’s own SMTP, it’s easy to redirect the emails from the local Django application being developed to this dummy mail server and see the emails on the console.
Here’s how to do it:

  • Run Python SMTP with this command line:
    python -m smtpd -n -c DebuggingServer localhost:1025
  • Define Django email server as:
    EMAIL_HOST = 'localhost'
    EMAIL_PORT = 1025

And that’s it. All emails sent will be seen on the console where you’r running the Python SMTP.
This technique is actually quite useful since you can use this for any application you are developing locally.

Redirect to Console

There’s another way to see the emails on the console with no dependencies.
To do so, just configure the email to use the console email backend.

EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

This will actually redirect all emails to the standard output, which usually is the console.

Redirect to File

Another approach is to redirect the emails into a local file for prior usage.
To do so, just configure the email to use the file email backend.

EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
EMAIL_FILE_PATH = '/tmp/app-messages' # change this to a proper location

This will write all emails into separate files located in

/tmp/app-messages

and you can analyze them latter.

Install and Configure PostgreSQL

 Database, Linux, PostgreSQL  Comments Off on Install and Configure PostgreSQL
May 052014
 

Here’s how to correctly install and configure PostgreSQL.

Install

First update the system

sudo apt-get update

Then install the packages:

sudo apt-get install postgresql postgresql-contrib

Create Database and User

Now let’s create a PostgreSQL roles and databases.

PostgreSQL uses the concept of roles to distinguish the variety of users that can connect to a database. After a fresh install, the default PostgreSQL user is actual named “postgres”.

Login into the “postgres” user using

sudo su postgres

and add a user

createuser -e -P USERNAME

set the password for the specified USERNAME.

Now create a database and assign the user to it

createdb -O USERNAME DATABASENAME

Connect to the database using

psql -d DATABASENAME -U USERNAME

And that’s it.

Correctly Uninstall PostgreSQL

 Database, Linux, PostgreSQL  Comments Off on Correctly Uninstall PostgreSQL
May 022014
 

Uninstalling PostgreSQL may leave some undesired files. Here’s how to purge it:

First, uninstall all packages from PostgreSQL from the system using

sudo apt-get --purge remove postgresql\*

Then remove all the configuration and library stuff

sudo rm -r /etc/postgresql/
sudo rm -r /etc/postgresql-common/
sudo rm -r /var/lib/postgresql/

And finally, remove the user and group

sudo userdel -r postgres
sudo groupdel postgres

Upgrade PostgreSQL from 9.1 to 9.3 on Kubuntu

 Data Migration, Database, Linux, PostgreSQL  Comments Off on Upgrade PostgreSQL from 9.1 to 9.3 on Kubuntu
Apr 192014
 

This seven steps will perform the upgrade of PostgreSQL from version 9.1 to version 9.3.
This also works in Ubuntu and it can also be used to upgrade between any version numbers.

To upgrade between any versions, just changed the 9.1 for the legacy version number and the 9.3 for the new version number.

First install the necessary dependencies

sudo apt-get update
sudo apt-get -y install python-software-properties

Second, add the PostgreSQL repository

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

Third, setup the repository

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" >> /etc/apt/sources.list.d/postgresql.list'

Fourth, install PostgreSQL 9.3.

sudo apt-get install postgresql-9.3 postgresql-server-dev-9.3 postgresql-contrib-9.3

Fifth, perform the upgrade process.
The upgrade process is performed by having both servers running at the same time. Note that the new 9.3 version will run on a different port, 5433 as specified in the script bellow, and it will be set to the default port latter when the legacy version is uninstalled and the default port becomes available.

sudo su -l postgres
psql -d template1 -p 5433
CREATE EXTENSION IF NOT EXISTS hstore;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
\q #logout from database
service postgresql stop
/usr/lib/postgresql/9.3/bin/pg_upgrade -b /usr/lib/postgresql/9.1/bin -B /usr/lib/postgresql/9.3/bin -d /var/lib/postgresql/9.1/main/ -D /var/lib/postgresql/9.3/main/ -O " -c config_file=/etc/postgresql/9.3/main/postgresql.conf" -o "-c config_file=/etc/postgresql/9.1/main/postgresql.conf"
exit # logout postgresql back to previous user

Sixth, remove the 9.1 version.

sudo apt-get remove postgresql-9.1

Seventh, set the new version server port back to the default value and restart the service.

sudo vim /etc/postgresql/9.3/main/postgresql.conf # find old port of 5433 and change it to 5432
sudo service postgresql restart

Change Keyboard Layout on Ubuntu Server Permanently

 Linux, Operating System  Comments Off on Change Keyboard Layout on Ubuntu Server Permanently
Mar 062014
 

When working on Ubuntu Servers, sometimes the defined keyboard layout is not aligned with the physical keyboard one’s using.

In order to permanently change it, just execute the following two commands.

First, configure the keyboard:

sudo dpkg-reconfigure keyboard-configuration

Test the keyboard, in particular the characters like slash, asterisk, etc.. If things are not ok, just configure it again with different options.

Next, configure the console:

sudo dpkg-reconfigure console-setup

And that’s it.

If you want to configure the size of your TTY, i. e. the text console resolution, check ChangeTTYResolution.

Testing for Internet Explorer

 Internet Explorer, Linux, Mac OS X, Operating System, Web, Windows  Comments Off on Testing for Internet Explorer
Nov 082013
 

While developing for the web, one has to take into account the various browsers that people use.
Depending on the kind of project and target users, this may range from an easy option focused on a single browser up to the dawnting task of supporting the most used browsers world wide.

To help in this task, Microsoft has provided a set of combinations of Windows and Internet Explorer versions in ready to use virtual machines available to all major operating systems.
You can get them here in the Modern.IE web site in the virtual tools section.

Setting Drupal File Permissions and Ownership

 Drupal, Linux, Operating System  Comments Off on Setting Drupal File Permissions and Ownership
Nov 042013
 

To correctly secure a Drupal installation in Linux, just follow these simple steps:

[root@localhost]cd /path_to_drupal_installation
[root@localhost]chown -R vsftpd:www-data .
[root@localhost]find . -type d -exec chmod u=rwx,g=rx,o= '{}' \;
[root@localhost]find . -type f -exec chmod u=rw,g=r,o= '{}' \;

Change the vsftpd by your own user name, ftp deamon user or what evere user you need.

[root@localhost]cd /path_to_drupal_installation/sites
[root@localhost]find . -type d -name files -exec chmod ug=rwx,o= '{}' \;
[root@localhost]for d in ./*/files
do
   find $d -type d -exec chmod ug=rwx,o= '{}' \;
   find $d -type f -exec chmod ug=rw,o= '{}' \;
done

Full article and explanation from Securing file permissions and ownership.

Print the result of a cursor in Oracle PL/SQL

 Database, Oracle, PL/SQL  Comments Off on Print the result of a cursor in Oracle PL/SQL
May 272013
 

While developing Oracle PL/SQL code, sometimes one needs to debug via the old way, using DBMS_OUTPUT.PUT_LINE or inserting data into a table for a later inspection.
But when working with cursors, this is not an easy task, specially if one does not know the real output of the cursor when using dynamic SQL or simply because the cost of writing a specific output for each cursor one needs to evaluate is too high.

In order to aid this task, here’s a procedure that will print out the result of any cursor. By default, it will print the first 10 rows, but to print the entire result, just call the PRINT_CURSOR procedure with the v_maxRows = 0.

create or replace
PROCEDURE PRINT_CURSOR (
    p_refcursor IN OUT SYS_REFCURSOR, v_maxRows IN NUMBER default 10 )
AS
  v_desc DBMS_SQL.DESC_TAB;
  v_cols BINARY_INTEGER;
  v_cursor BINARY_INTEGER;
  v_varchar2 VARCHAR2(4000);
  v_number   NUMBER;
  v_date     DATE;
  v_data     VARCHAR2(32767);
  v_curRow   NUMBER;
BEGIN
  /* Convert refcursor "parameter" to DBMS_SQL cursor... */
  v_cursor := DBMS_SQL.TO_CURSOR_NUMBER(p_refcursor);
  /* Describe the cursor... */
  DBMS_SQL.DESCRIBE_COLUMNS(v_cursor, v_cols, v_desc);
  /* Define columns to be fetched. We're only using V2, NUM, DATE for example... */
  FOR i IN 1 .. v_cols
  LOOP
    IF v_desc(i).col_type = 2 THEN
      DBMS_SQL.DEFINE_COLUMN(v_cursor, i, v_number);
    ELSIF v_desc(i).col_type = 12 THEN
      DBMS_SQL.DEFINE_COLUMN(v_cursor, i, v_date);
    ELSE
      DBMS_SQL.DEFINE_COLUMN(v_cursor, i, v_varchar2, 4000);
    END IF;
  END LOOP;
  /* Now output the data, starting with header... */
  DBMS_OUTPUT.NEW_LINE;
  FOR i IN 1 .. v_cols
  LOOP
    v_data := v_data ||
    CASE v_desc(i).col_type
    WHEN 2 THEN
      LPAD(v_desc(i).col_name, v_desc(i).col_max_len+1)
    WHEN 12 THEN
      RPAD(v_desc(i).col_name, 22)
    ELSE
      RPAD(v_desc(i).col_name, v_desc(i).col_max_len+1)
    END || ' ';
  END LOOP;
  DBMS_OUTPUT.PUT_LINE(v_data);
  v_data := NULL;
  FOR i IN 1 .. v_cols
  LOOP
    v_data := v_data ||
    CASE v_desc(i).col_type
    WHEN 2 THEN
      LPAD('-', v_desc(i).col_max_len+1, '-')
    WHEN 12 THEN
      RPAD('-', 22, '-')
    ELSE
      RPAD('-', v_desc(i).col_max_len+1, '-')
    END || ' ';
  END LOOP;
  DBMS_OUTPUT.PUT_LINE(v_data);
  /* Fetch all data... */
  v_curRow := v_maxRows;
  WHILE DBMS_SQL.FETCH_ROWS(v_cursor) > 0
  LOOP
    v_data := NULL;
    FOR i IN 1 .. v_cols
    LOOP
      IF v_desc(i).col_type = 2 THEN
        DBMS_SQL.COLUMN_VALUE(v_cursor, i, v_number);
        v_data                := v_data || LPAD(v_number, v_desc(i).col_max_len+1) || ' ';
      ELSIF v_desc(i).col_type = 12 THEN
        DBMS_SQL.COLUMN_VALUE(v_cursor, i, v_date);
        v_data := v_data || RPAD(v_date, 22) || ' ';
      ELSE
        DBMS_SQL.COLUMN_VALUE(v_cursor, i, v_varchar2);
        v_data := v_data || RPAD(v_varchar2, v_desc(i).col_max_len+1) || ' ';
      END IF;
    END LOOP;
    DBMS_OUTPUT.PUT_LINE(v_data);
    if  v_maxRows <> 0 then
       v_curRow :=  v_curRow - 1;
       exit when 0 = v_curRow;
    end if;
  END LOOP;
  DBMS_SQL.CLOSE_CURSOR(v_cursor);
END;

Enable and Disable constraints and triggers in Oracle

 Database, Oracle, PL/SQL  Comments Off on Enable and Disable constraints and triggers in Oracle
Apr 222013
 

During data migrations it’s common to have constraints disabled, usually for loading performance or testing purposes.
To aid in such task, we’re sharing two simple scripts, one for disabling and another for enabling.

To disable all enabled constraints and triggers, just execute:

BEGIN
  -- Constraints
  FOR c IN
  (SELECT c.owner, c.table_name, c.constraint_name
   FROM user_constraints c, user_tables t
   WHERE c.table_name = t.table_name
   AND c.status = 'ENABLED'
   ORDER BY c.constraint_type DESC)
  LOOP
    -- DISABLE
    execute immediate 'alter table "' || c.owner || '"."' || c.table_name || '" disable constraint ' || c.constraint_name;
  END LOOP;
 
  -- Triggers
  FOR ut IN
    (select ut.table_owner, ut.trigger_name
    from user_triggers ut
    where ut.status = 'ENABLED')
  LOOP
    -- DISABLE
    execute immediate 'ALTER TRIGGER "' || ut.table_owner || '"."' || ut.trigger_name || '" DISABLE;';
  END LOOP;
END;
/

To enable all disabled constraints and triggers, just execute:

BEGIN
  -- Constraints
  FOR c IN
  (SELECT c.owner, c.table_name, c.constraint_name
   FROM user_constraints c, user_tables t
   WHERE c.table_name = t.table_name
   AND c.status = 'DISABLE'
   ORDER BY c.constraint_type DESC)
  LOOP
    -- ENABLE
    execute immediate 'alter table "' || c.owner || '"."' || c.table_name || '" enable constraint ' || c.constraint_name;
  END LOOP;
 
  -- Triggers
  FOR ut IN
    (select ut.table_owner, ut.trigger_name
    from user_triggers ut
    where ut.status = 'DISABLE')
  LOOP
    -- ENABLE
    execute immediate 'ALTER TRIGGER "' || ut.table_owner || '"."' || ut.trigger_name || '" ENABLE;';
  END LOOP;
END;
/

For XML Path Equivalent in Oracle For String Aggregation

 Data Migration, Database, ETL, Oracle  Comments Off on For XML Path Equivalent in Oracle For String Aggregation
Oct 312012
 

During a migration from SQL Server to Oracle 11g, I’ve came across with a specific functionality of T-SQL FOR XML PATH.
In the scope of the migration, the result of the select clause where this was applied was the concatenation of the values of a column per key.

Here’s an example of what I’ve found on T-SQL during the migration:

Declare @tbl1 table (
  id int,
  Col1 varchar(50),
  Col2 varchar(50)
)

INSERT INTO @tbl1
SELECT 1,'A001','Y'

INSERT INTO @tbl1
SELECT 2,'A002','N'

INSERT INTO @tbl1
SELECT 3,'A003','N'

INSERT INTO @tbl1
SELECT 4,'A004','Y'

INSERT INTO @tbl1
SELECT 5,'A005','N'

INSERT INTO @tbl1
SELECT 6,'A006','N'

INSERT INTO @tbl1
SELECT 7,'A007','N'

INSERT INTO @tbl1
SELECT 8,'A008','Y'

-- Show current values
select * from @tbl1

-- Get all values when col2 has value 'N'
SELECT c.col1 AS [text()]
FROM @tbl1 c
WHERE c.col2 = 'N'
FOR XML PATH(''),type

-- Aggregate all values from col1 using col2 as key
select c1.col2 [Yes/No],
  (SELECT c.col1 AS [text()]
   FROM @tbl1 c
   WHERE c.col2 = c1.col2
   FOR XML PATH(''),type) as Value
FROM @tbl1 c1
group by c1.col2
order by c1.col2

The execution of this T-SQL script shows how it is possible transform the result of a query into a new result where the values are aggregated by col1, and all col2 values are concatenated.

In order to achieve this kind of aggregate transformation in Oracle, one has to make use of an associative array, as Tom explains in the “String Concatenation” how one can do this. Using the examples in the AskTom web site, first we create the “STRING_AGG_TYPE” type:

CREATE OR REPLACE TYPE string_agg_type AS object
(
  total VARCHAR2(4000),

  static FUNCTION ODCIAggregateInitialize(sctx IN OUT string_agg_type) RETURN NUMBER,

  member FUNCTION ODCIAggregateIterate(self IN OUT string_agg_type, VALUE IN VARCHAR2) RETURN NUMBER,

  member FUNCTION ODCIAggregateTerminate(self IN string_agg_type, returnValue OUT VARCHAR2, flags IN NUMBER) RETURN NUMBER,

  member FUNCTION ODCIAggregateMerge(self IN OUT string_agg_type, ctx2 IN string_agg_type) RETURN NUMBER
);
/

And the correspondent body:

CREATE OR REPLACE TYPE BODY string_agg_type
IS

static FUNCTION ODCIAggregateInitialize(sctx IN OUT string_agg_type)
RETURN NUMBER
IS
BEGIN
  sctx := string_agg_type( NULL );
  RETURN ODCIConst.Success;
END;

member FUNCTION ODCIAggregateIterate(self IN OUT string_agg_type, VALUE IN VARCHAR2 )
RETURN NUMBER
IS
BEGIN
  -- NOTE: Insert a separator in the empty string if you wish
  self.total := self.total || '' || VALUE;
  RETURN ODCIConst.Success;
END;

member FUNCTION ODCIAggregateTerminate(self IN string_agg_type, returnValue OUT VARCHAR2, flags IN NUMBER)
RETURN NUMBER
IS
BEGIN
  -- NOTE: ltrim(self.total, '') if you've specified a separator in ODCIAggregateIterate
  returnValue := LTRIM(self.total);
  RETURN ODCIConst.Success;
END;

member FUNCTION ODCIAggregateMerge(self IN OUT string_agg_type, ctx2 IN string_agg_type)
RETURN NUMBER
IS
BEGIN
  self.total := self.total || ctx2.total;
  RETURN ODCIConst.Success;
END;

END;
/

Then we create the aggregated function “stragg” of the type we’ve just created:

CREATE OR REPLACE FUNCTION stragg(input VARCHAR2 ) RETURN VARCHAR2 PARALLEL_ENABLE AGGREGATE USING string_agg_type;
/

We are now able to make a similar aggregation in Oracle:

CREATE TABLE tbl1 (
  id NUMBER(10),
  Col1 VARCHAR(50),
  Col2 VARCHAR(50)
);
/

INSERT INTO tbl1 (id, col1, col2)
VALUES (1,'A001','Y');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (2,'A002','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (3,'A003','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (4,'A004','Y');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (5,'A005','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (6,'A006','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (7,'A007','N');
/
INSERT INTO tbl1 (id, col1, col2)
VALUES (8,'A008','Y');
/
COMMIT;
/

-- Show current values
SELECT * FROM tbl1;
/

-- Get all values when col2 has value 'N'
SELECT c.col2, STRAGG(c.col1)
FROM tbl1 c
WHERE c.col2 = 'N'
GROUP BY c.col2;
/

-- Aggregate all values from col1 using col2 as key
SELECT c1.col2 "Yes/No",
  (SELECT STRAGG(c.col1)
   FROM tbl1 c
   WHERE c.col2 = c1.col2
  ) AS val
FROM tbl1 c1
GROUP BY c1.col2
ORDER BY c1.col2;
/

-- drop table tbl1;

And that’s it. This will work in 9i and beyond, for 8i, check the 8i workaround in the same AskTom thread.