/usr/share/doc/postgresql-9.4-plproxy/faq.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
    "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
<meta name="generator" content="AsciiDoc 8.6.9" />
<title>PL/Proxy FAQ</title>
<style type="text/css">
/* Shared CSS for AsciiDoc xhtml11 and html5 backends */

/* Default font. */
body {
  font-family: Georgia,serif;
}

/* Title font. */
h1, h2, h3, h4, h5, h6,
div.title, caption.title,
thead, p.table.header,
#toctitle,
#author, #revnumber, #revdate, #revremark,
#footer {
  font-family: Arial,Helvetica,sans-serif;
}

body {
  margin: 1em 5% 1em 5%;
}

a {
  color: blue;
  text-decoration: underline;
}
a:visited {
  color: fuchsia;
}

em {
  font-style: italic;
  color: navy;
}

strong {
  font-weight: bold;
  color: #083194;
}

h1, h2, h3, h4, h5, h6 {
  color: #527bbd;
  margin-top: 1.2em;
  margin-bottom: 0.5em;
  line-height: 1.3;
}

h1, h2, h3 {
  border-bottom: 2px solid silver;
}
h2 {
  padding-top: 0.5em;
}
h3 {
  float: left;
}
h3 + * {
  clear: left;
}
h5 {
  font-size: 1.0em;
}

div.sectionbody {
  margin-left: 0;
}

hr {
  border: 1px solid silver;
}

p {
  margin-top: 0.5em;
  margin-bottom: 0.5em;
}

ul, ol, li > p {
  margin-top: 0;
}
ul > li     { color: #aaa; }
ul > li > * { color: black; }

.monospaced, code, pre {
  font-family: "Courier New", Courier, monospace;
  font-size: inherit;
  color: navy;
  padding: 0;
  margin: 0;
}
pre {
  white-space: pre-wrap;
}

#author {
  color: #527bbd;
  font-weight: bold;
  font-size: 1.1em;
}
#email {
}
#revnumber, #revdate, #revremark {
}

#footer {
  font-size: small;
  border-top: 2px solid silver;
  padding-top: 0.5em;
  margin-top: 4.0em;
}
#footer-text {
  float: left;
  padding-bottom: 0.5em;
}
#footer-badges {
  float: right;
  padding-bottom: 0.5em;
}

#preamble {
  margin-top: 1.5em;
  margin-bottom: 1.5em;
}
div.imageblock, div.exampleblock, div.verseblock,
div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
div.admonitionblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
div.admonitionblock {
  margin-top: 2.0em;
  margin-bottom: 2.0em;
  margin-right: 10%;
  color: #606060;
}

div.content { /* Block element content. */
  padding: 0;
}

/* Block element titles. */
div.title, caption.title {
  color: #527bbd;
  font-weight: bold;
  text-align: left;
  margin-top: 1.0em;
  margin-bottom: 0.5em;
}
div.title + * {
  margin-top: 0;
}

td div.title:first-child {
  margin-top: 0.0em;
}
div.content div.title:first-child {
  margin-top: 0.0em;
}
div.content + div.title {
  margin-top: 0.0em;
}

div.sidebarblock > div.content {
  background: #ffffee;
  border: 1px solid #dddddd;
  border-left: 4px solid #f0f0f0;
  padding: 0.5em;
}

div.listingblock > div.content {
  border: 1px solid #dddddd;
  border-left: 5px solid #f0f0f0;
  background: #f8f8f8;
  padding: 0.5em;
}

div.quoteblock, div.verseblock {
  padding-left: 1.0em;
  margin-left: 1.0em;
  margin-right: 10%;
  border-left: 5px solid #f0f0f0;
  color: #888;
}

div.quoteblock > div.attribution {
  padding-top: 0.5em;
  text-align: right;
}

div.verseblock > pre.content {
  font-family: inherit;
  font-size: inherit;
}
div.verseblock > div.attribution {
  padding-top: 0.75em;
  text-align: left;
}
/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
div.verseblock + div.attribution {
  text-align: left;
}

div.admonitionblock .icon {
  vertical-align: top;
  font-size: 1.1em;
  font-weight: bold;
  text-decoration: underline;
  color: #527bbd;
  padding-right: 0.5em;
}
div.admonitionblock td.content {
  padding-left: 0.5em;
  border-left: 3px solid #dddddd;
}

div.exampleblock > div.content {
  border-left: 3px solid #dddddd;
  padding-left: 0.5em;
}

div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; vertical-align: text-bottom; }
a.image:visited { color: white; }

dl {
  margin-top: 0.8em;
  margin-bottom: 0.8em;
}
dt {
  margin-top: 0.5em;
  margin-bottom: 0;
  font-style: normal;
  color: navy;
}
dd > *:first-child {
  margin-top: 0.1em;
}

ul, ol {
    list-style-position: outside;
}
ol.arabic {
  list-style-type: decimal;
}
ol.loweralpha {
  list-style-type: lower-alpha;
}
ol.upperalpha {
  list-style-type: upper-alpha;
}
ol.lowerroman {
  list-style-type: lower-roman;
}
ol.upperroman {
  list-style-type: upper-roman;
}

div.compact ul, div.compact ol,
div.compact p, div.compact p,
div.compact div, div.compact div {
  margin-top: 0.1em;
  margin-bottom: 0.1em;
}

tfoot {
  font-weight: bold;
}
td > div.verse {
  white-space: pre;
}

div.hdlist {
  margin-top: 0.8em;
  margin-bottom: 0.8em;
}
div.hdlist tr {
  padding-bottom: 15px;
}
dt.hdlist1.strong, td.hdlist1.strong {
  font-weight: bold;
}
td.hdlist1 {
  vertical-align: top;
  font-style: normal;
  padding-right: 0.8em;
  color: navy;
}
td.hdlist2 {
  vertical-align: top;
}
div.hdlist.compact tr {
  margin: 0;
  padding-bottom: 0;
}

.comment {
  background: yellow;
}

.footnote, .footnoteref {
  font-size: 0.8em;
}

span.footnote, span.footnoteref {
  vertical-align: super;
}

#footnotes {
  margin: 20px 0 20px 0;
  padding: 7px 0 0 0;
}

#footnotes div.footnote {
  margin: 0 0 5px 0;
}

#footnotes hr {
  border: none;
  border-top: 1px solid silver;
  height: 1px;
  text-align: left;
  margin-left: 0;
  width: 20%;
  min-width: 100px;
}

div.colist td {
  padding-right: 0.5em;
  padding-bottom: 0.3em;
  vertical-align: top;
}
div.colist td img {
  margin-top: 0.3em;
}

@media print {
  #footer-badges { display: none; }
}

#toc {
  margin-bottom: 2.5em;
}

#toctitle {
  color: #527bbd;
  font-size: 1.1em;
  font-weight: bold;
  margin-top: 1.0em;
  margin-bottom: 0.1em;
}

div.toclevel0, div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
  margin-top: 0;
  margin-bottom: 0;
}
div.toclevel2 {
  margin-left: 2em;
  font-size: 0.9em;
}
div.toclevel3 {
  margin-left: 4em;
  font-size: 0.9em;
}
div.toclevel4 {
  margin-left: 6em;
  font-size: 0.9em;
}

span.aqua { color: aqua; }
span.black { color: black; }
span.blue { color: blue; }
span.fuchsia { color: fuchsia; }
span.gray { color: gray; }
span.green { color: green; }
span.lime { color: lime; }
span.maroon { color: maroon; }
span.navy { color: navy; }
span.olive { color: olive; }
span.purple { color: purple; }
span.red { color: red; }
span.silver { color: silver; }
span.teal { color: teal; }
span.white { color: white; }
span.yellow { color: yellow; }

span.aqua-background { background: aqua; }
span.black-background { background: black; }
span.blue-background { background: blue; }
span.fuchsia-background { background: fuchsia; }
span.gray-background { background: gray; }
span.green-background { background: green; }
span.lime-background { background: lime; }
span.maroon-background { background: maroon; }
span.navy-background { background: navy; }
span.olive-background { background: olive; }
span.purple-background { background: purple; }
span.red-background { background: red; }
span.silver-background { background: silver; }
span.teal-background { background: teal; }
span.white-background { background: white; }
span.yellow-background { background: yellow; }

span.big { font-size: 2em; }
span.small { font-size: 0.6em; }

span.underline { text-decoration: underline; }
span.overline { text-decoration: overline; }
span.line-through { text-decoration: line-through; }

div.unbreakable { page-break-inside: avoid; }


/*
 * xhtml11 specific
 *
 * */

div.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
div.tableblock > table {
  border: 3px solid #527bbd;
}
thead, p.table.header {
  font-weight: bold;
  color: #527bbd;
}
p.table {
  margin-top: 0;
}
/* Because the table frame attribute is overriden by CSS in most browsers. */
div.tableblock > table[frame="void"] {
  border-style: none;
}
div.tableblock > table[frame="hsides"] {
  border-left-style: none;
  border-right-style: none;
}
div.tableblock > table[frame="vsides"] {
  border-top-style: none;
  border-bottom-style: none;
}


/*
 * html5 specific
 *
 * */

table.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
thead, p.tableblock.header {
  font-weight: bold;
  color: #527bbd;
}
p.tableblock {
  margin-top: 0;
}
table.tableblock {
  border-width: 3px;
  border-spacing: 0px;
  border-style: solid;
  border-color: #527bbd;
  border-collapse: collapse;
}
th.tableblock, td.tableblock {
  border-width: 1px;
  padding: 4px;
  border-style: solid;
  border-color: #527bbd;
}

table.tableblock.frame-topbot {
  border-left-style: hidden;
  border-right-style: hidden;
}
table.tableblock.frame-sides {
  border-top-style: hidden;
  border-bottom-style: hidden;
}
table.tableblock.frame-none {
  border-style: hidden;
}

th.tableblock.halign-left, td.tableblock.halign-left {
  text-align: left;
}
th.tableblock.halign-center, td.tableblock.halign-center {
  text-align: center;
}
th.tableblock.halign-right, td.tableblock.halign-right {
  text-align: right;
}

th.tableblock.valign-top, td.tableblock.valign-top {
  vertical-align: top;
}
th.tableblock.valign-middle, td.tableblock.valign-middle {
  vertical-align: middle;
}
th.tableblock.valign-bottom, td.tableblock.valign-bottom {
  vertical-align: bottom;
}


/*
 * manpage specific
 *
 * */

body.manpage h1 {
  padding-top: 0.5em;
  padding-bottom: 0.5em;
  border-top: 2px solid silver;
  border-bottom: 2px solid silver;
}
body.manpage h2 {
  border-style: none;
}
body.manpage div.sectionbody {
  margin-left: 3em;
}

@media print {
  body.manpage div#toc { display: none; }
}


</style>
<script type="text/javascript">
/*<![CDATA[*/
var asciidoc = {  // Namespace.

/////////////////////////////////////////////////////////////////////
// Table Of Contents generator
/////////////////////////////////////////////////////////////////////

/* Author: Mihai Bazon, September 2002
 * http://students.infoiasi.ro/~mishoo
 *
 * Table Of Content generator
 * Version: 0.4
 *
 * Feel free to use this script under the terms of the GNU General Public
 * License, as long as you do not remove or alter this notice.
 */

 /* modified by Troy D. Hanson, September 2006. License: GPL */
 /* modified by Stuart Rackham, 2006, 2009. License: GPL */

// toclevels = 1..4.
toc: function (toclevels) {

  function getText(el) {
    var text = "";
    for (var i = el.firstChild; i != null; i = i.nextSibling) {
      if (i.nodeType == 3 /* Node.TEXT_NODE */) // IE doesn't speak constants.
        text += i.data;
      else if (i.firstChild != null)
        text += getText(i);
    }
    return text;
  }

  function TocEntry(el, text, toclevel) {
    this.element = el;
    this.text = text;
    this.toclevel = toclevel;
  }

  function tocEntries(el, toclevels) {
    var result = new Array;
    var re = new RegExp('[hH]([1-'+(toclevels+1)+'])');
    // Function that scans the DOM tree for header elements (the DOM2
    // nodeIterator API would be a better technique but not supported by all
    // browsers).
    var iterate = function (el) {
      for (var i = el.firstChild; i != null; i = i.nextSibling) {
        if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
          var mo = re.exec(i.tagName);
          if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
            result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
          }
          iterate(i);
        }
      }
    }
    iterate(el);
    return result;
  }

  var toc = document.getElementById("toc");
  if (!toc) {
    return;
  }

  // Delete existing TOC entries in case we're reloading the TOC.
  var tocEntriesToRemove = [];
  var i;
  for (i = 0; i < toc.childNodes.length; i++) {
    var entry = toc.childNodes[i];
    if (entry.nodeName.toLowerCase() == 'div'
     && entry.getAttribute("class")
     && entry.getAttribute("class").match(/^toclevel/))
      tocEntriesToRemove.push(entry);
  }
  for (i = 0; i < tocEntriesToRemove.length; i++) {
    toc.removeChild(tocEntriesToRemove[i]);
  }

  // Rebuild TOC entries.
  var entries = tocEntries(document.getElementById("content"), toclevels);
  for (var i = 0; i < entries.length; ++i) {
    var entry = entries[i];
    if (entry.element.id == "")
      entry.element.id = "_toc_" + i;
    var a = document.createElement("a");
    a.href = "#" + entry.element.id;
    a.appendChild(document.createTextNode(entry.text));
    var div = document.createElement("div");
    div.appendChild(a);
    div.className = "toclevel" + entry.toclevel;
    toc.appendChild(div);
  }
  if (entries.length == 0)
    toc.parentNode.removeChild(toc);
},


/////////////////////////////////////////////////////////////////////
// Footnotes generator
/////////////////////////////////////////////////////////////////////

/* Based on footnote generation code from:
 * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
 */

footnotes: function () {
  // Delete existing footnote entries in case we're reloading the footnodes.
  var i;
  var noteholder = document.getElementById("footnotes");
  if (!noteholder) {
    return;
  }
  var entriesToRemove = [];
  for (i = 0; i < noteholder.childNodes.length; i++) {
    var entry = noteholder.childNodes[i];
    if (entry.nodeName.toLowerCase() == 'div' && entry.getAttribute("class") == "footnote")
      entriesToRemove.push(entry);
  }
  for (i = 0; i < entriesToRemove.length; i++) {
    noteholder.removeChild(entriesToRemove[i]);
  }

  // Rebuild footnote entries.
  var cont = document.getElementById("content");
  var spans = cont.getElementsByTagName("span");
  var refs = {};
  var n = 0;
  for (i=0; i<spans.length; i++) {
    if (spans[i].className == "footnote") {
      n++;
      var note = spans[i].getAttribute("data-note");
      if (!note) {
        // Use [\s\S] in place of . so multi-line matches work.
        // Because JavaScript has no s (dotall) regex flag.
        note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
        spans[i].innerHTML =
          "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
          "' title='View footnote' class='footnote'>" + n + "</a>]";
        spans[i].setAttribute("data-note", note);
      }
      noteholder.innerHTML +=
        "<div class='footnote' id='_footnote_" + n + "'>" +
        "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
        n + "</a>. " + note + "</div>";
      var id =spans[i].getAttribute("id");
      if (id != null) refs["#"+id] = n;
    }
  }
  if (n == 0)
    noteholder.parentNode.removeChild(noteholder);
  else {
    // Process footnoterefs.
    for (i=0; i<spans.length; i++) {
      if (spans[i].className == "footnoteref") {
        var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
        href = href.match(/#.*/)[0];  // Because IE return full URL.
        n = refs[href];
        spans[i].innerHTML =
          "[<a href='#_footnote_" + n +
          "' title='View footnote' class='footnote'>" + n + "</a>]";
      }
    }
  }
},

install: function(toclevels) {
  var timerId;

  function reinstall() {
    asciidoc.footnotes();
    if (toclevels) {
      asciidoc.toc(toclevels);
    }
  }

  function reinstallAndRemoveTimer() {
    clearInterval(timerId);
    reinstall();
  }

  timerId = setInterval(reinstall, 500);
  if (document.addEventListener)
    document.addEventListener("DOMContentLoaded", reinstallAndRemoveTimer, false);
  else
    window.onload = reinstallAndRemoveTimer;
}

}
asciidoc.install(2);
/*]]>*/
</script>
</head>
<body class="article">
<div id="header">
<h1>PL/Proxy FAQ</h1>
<div id="toc">
  <div id="toctitle">Table of Contents</div>
  <noscript><p><b>JavaScript must be enabled in your browser to display the table of contents.</b></p></noscript>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="_general">General</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_what_is_pl_proxy">What is PL/Proxy?</h3>
<div class="paragraph"><p>PL/Proxy is compact language for remote calls between PostgreSQL
databases. Syntax is similar to PL/pgSql and language contains only 4
statements.</p></div>
<div class="paragraph"><p>With PL/Proxy user can create proxy functions that have same signature
as remote functions to be called.  The function body describes
how the remote connection should be acquired.</p></div>
<div class="paragraph"><p>When such proxy function is called, PL/Proxy:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Automatically generates the SQL query to be executed from function signature
</p>
</li>
<li>
<p>
Executes statements in function body to get the connection
</p>
</li>
<li>
<p>
Uses function arguments as input parameters to query
</p>
</li>
<li>
<p>
Passes the query result back as function result
</p>
</li>
</ol></div>
</div>
<div class="sect2">
<h3 id="_why_functions">Why functions?</h3>
<div class="paragraph"><p>Concentrating on just function-calls allows PL/Proxy to keep
its code small and also to present user simple and compact API.</p></div>
<div class="ulist"><ul>
<li>
<p>
The actual query run on remote database can be generated
  based on plproxy function signature.  User just needs to
  specify how the connection to remote database must be acquired.
</p>
</li>
<li>
<p>
There is no need for complex transaction handling as any
  multi-statement transactions can be put into functions.
  PL/Proxy can just execute all queries in <em>autocommit</em> mode.
</p>
</li>
<li>
<p>
Simple autocommit transactions mean that the connection
  handling is simple and can be done automatically.
</p>
</li>
</ul></div>
<div class="paragraph"><p>Using function-based database access has more general good points:</p></div>
<div class="ulist"><ul>
<li>
<p>
It&#8217;s good to have SQL statements that operate on data
  near to tables.  That makes life of DBA&#8217;s easier.
</p>
</li>
<li>
<p>
It makes it possible to optimize and reorganize tables
  transparently to the application.
</p>
</li>
<li>
<p>
Enables DBA&#8217;s to move parts of database into another
  database without changing application interface.
</p>
</li>
<li>
<p>
Easier to manage security if you don&#8217;t have to do it on
  table level.  In most cases you need to control what user
  can do and on which data not on what tables.
</p>
</li>
<li>
<p>
All transactions can be made in <em>autocommit</em> mode.
  That means absolutely minimal amount of roundtrips (1)
  for each query and also each transaction takes shortest
  possible amount of time on server - remember that various
  locks that transactions aquire are released on COMMIT.
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_why_not_develop_it_into_remote_parallel_pl_sql">Why not develop it into Remote Parallel PL/SQL?</h3>
<div class="paragraph"><p>Huge benefit of PL/Proxy is it&#8217;s compactness and efficiency.
As it does not need to parse queries going through it adds very
little overhead.</p></div>
<div class="paragraph"><p>Making it full-blown language for SQL execution would mean
reimplementing PL/pgSQL, PL/Perl, parts of pgpool and more,
which is waste of effort.</p></div>
<div class="paragraph"><p>Also when plproxy functions mirror actual functions, the
PL/Proxy becomes optional component of setup - the client
apps can bypass PL/Proxy and work directly on actual database.</p></div>
<div class="paragraph"><p>This is good for testing and also live deployment - we let
clients work on smaller databases directly, they are put
behind PL/Proxy only when load gets too high and we need
to partition a database.</p></div>
</div>
<div class="sect2">
<h3 id="_what_can_pl_proxy_be_used_for">What can PL/Proxy be used for?</h3>
<div class="ulist"><ul>
<li>
<p>
Remote calls from one database to another either used inside SQL or other procedures.
  (If used as part of local transaction need to make sure only
   one side is allowed to write to database, PL/Proxy does not
   guarantee transactionality between 2 databases.)
</p>
</li>
<li>
<p>
Proxy databases for better security and data protection.
</p>
</li>
<li>
<p>
Proxy databases for hiding complexity of databases from application,
  eg. if you have functions distributed between several databases
</p>
</li>
<li>
<p>
Horizontal partitioning.  Instead of buying more powerful servers you can
  split your data between several servers and then use PL/Proxy to redirect
  function calls into right partitions.
</p>
</li>
<li>
<p>
Load balancing if you have several read only replicas of your data.
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_how_does_it_compare_to_dblink">How does it compare to dblink?</h3>
<div class="ulist"><ul>
<li>
<p>
PL/Proxy handles connections automatically, dblink forces user to handle them.
</p>
</li>
<li>
<p>
PL/Proxy has single place where result column types are specified - function signature.
  dblink requires them to be specified in each query.
</p>
</li>
<li>
<p>
PL/Proxy makes easy to run a query in several remote servers in parallel.
  Seems that dblink async API makes that also possible, but the usage is complex.
</p>
</li>
<li>
<p>
dblink allows arbitrary complex transactions, PL/Proxy runs everything
  in autocommit mode.  As previourly discussed, when using functions the
  complex transactions are not needed and with such tradeoff PL/Proxy can
  offer much simpler API.
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_how_are_pl_proxy_and_pgbouncer_related">How are PL/Proxy and PgBouncer related?</h3>
<div class="paragraph"><p>PL/Proxy version 1 had PL and pooler integrated.  But such design
caused a lot of unnecessary complexity.  With PL/Proxy version 2,
we wrote both pooler and PL part from scratch, both designed
to be standalone components.</p></div>
<div class="paragraph"><p>That allowed both components to be tested and used separately
and resulted in compact and robust codebase.</p></div>
<div class="paragraph"><p>So PgBouncer can be used with PL/Proxy to lessen connection count
on partition server, but such usage is not mandatory.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_internals">Internals</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_what_are_the_external_dependencies">What are the external dependencies?</h3>
<div class="paragraph"><p>It depends only on libpq and poll(2) + gettimeofday(2) system calls.
So it should be quite portable.</p></div>
</div>
<div class="sect2">
<h3 id="_how_the_remote_calls_are_done">How the remote calls are done?</h3>
<div class="paragraph"><p>First a SELECT query is generated based on PL/Proxy function
signature.</p></div>
<div class="paragraph"><p>A function signature of:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>CREATE FUNCTION get_data(IN first_name text, IN last_name text, OUT bdate date, OUT balance numeric(20,10))</code></pre>
</div></div>
<div class="paragraph"><p>Results in following query:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>SELECT bdate::date, balance::numeric(20,10) FROM public.get_data($1::text, $2::text);</code></pre>
</div></div>
<div class="paragraph"><p>The casts and explicit <code>OUT</code> parameter names are used to survive minor type or
result column order differences between local and remote databases.</p></div>
<div class="paragraph"><p>Then the <code>CLUSTER</code> statement is processed, optionally
executing function.  This result in cluster name.</p></div>
<div class="paragraph"><p>Then <code>plproxy.get_cluster_version(&lt;cluster_name&gt;)</code>
is executed.  This gives numeric version number for cluster.
If resulting version number differs from version
in cached cluster, the cache is dropped.</p></div>
<div class="paragraph"><p>If cluster information in not cached, the <code>plproxy.get_cluster_partitions()</code>
function is executed, resulting in list of connect strings for that cluster.</p></div>
<div class="paragraph"><p>Then <code>RUN</code> statement is processed, optionally executing
function if requested.  This will tag one or more connections
in cluster to be used for query execution.</p></div>
<div class="paragraph"><p>Then the query is sent to remote server using libpq async API.
If there are several remote connections tagged, the execution
will happen in parallel.  PL/Proxy then waits until it has
acquired resultsets from all connections and then returns
them to local backend.</p></div>
</div>
<div class="sect2">
<h3 id="_how_does_pl_proxy_handle_connections">How does PL/Proxy handle connections?</h3>
<div class="paragraph"><p>It opens them lazily - only when needed.  Then keeps them
open until it libpq reports error on it or connection
lifetime is over - which is by default 2h.</p></div>
<div class="paragraph"><p>There is a safety hack used - before sending query
to already open connection a poll(2) call is run on
connection.  If poll() shows events the connection
is dropped to avoid use of likely broken connection.</p></div>
</div>
<div class="sect2">
<h3 id="_can_pl_proxy_survive_different_settings_in_local_and_remote_database">Can PL/Proxy survive different settings in local and remote database?</h3>
<div class="dlist"><dl>
<dt class="hdlist1">
client_encoding
</dt>
<dd>
<p>
  If it differs, PL/Proxy sets the <code>client_encoding</code> on remote database
  to be equal to local one.
</p>
</dd>
<dt class="hdlist1">
standard_conforming_strings
</dt>
<dd>
<p>
  Query parameters are passed separately, so in general the difference
  should not matter.  Except when function uses explicit SELECT
  and it contains literal strings.  Fix is to avoid use of SELECT.
</p>
</dd>
<dt class="hdlist1">
datestyle, timezone
</dt>
<dd>
<p>
  Currently no handling is done.
</p>
</dd>
<dt class="hdlist1">
Rest of parameters
</dt>
<dd>
<p>
  Cannot be handled.
</p>
</dd>
</dl></div>
</div>
<div class="sect2">
<h3 id="_why_does_pl_proxy_require_the_number_of_partition_be_power_of_2">Why does PL/Proxy require the number of partition be power of 2?</h3>
<div class="paragraph"><p>There is no deep reason, mostly because of following points:</p></div>
<div class="ulist"><ul>
<li>
<p>
To have minimal sanity-checking on the output of get_cluster_partitions().
</p>
</li>
<li>
<p>
To have clear way to map hashes to partition.  As users quite
  likely need to write their own code for splitting and sanity checking
  their data, the algorithm should be as simple as possible.
</p>
</li>
</ul></div>
<div class="paragraph"><p>It would be easy to use mod N internally, but:</p></div>
<div class="ulist"><ul>
<li>
<p>
We would lose the sanity checking.
</p>
</li>
<li>
<p>
We would need to define mod function for negative integers that
  maps to positive range.  This sounds like a source for confusion and bugs.
</p>
</li>
</ul></div>
<div class="paragraph"><p>So it seems it&#8217;s preferable to keep the power-of-2 requirement.</p></div>
<div class="paragraph"><p>This may seem to require that the number of servers be also power of 2,
but this is not so - to make future administration easier it is
always preferable to split database into more parts than you
immediately need.  Such splitting also overcomes the power-of-2
requirement.</p></div>
<div class="paragraph"><p>For example, if user needs to spread the load over 3 servers,
the database can be split to 16 partitions and then 2 servers
get 5 partitions and last one 6.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_partitioning">Partitioning</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_how_to_partition_data">How to partition data?</h3>
<div class="paragraph"><p>There are several usage patterns how PL/Proxy can be used
to distribute load on several servers</p></div>
<div class="ulist"><ul>
<li>
<p>
Vertical partitioning.  Data is divided into separate servers table by table
  and PL/Proxy calls are used to direct calls to right databases.  In some cases
  wrapper functions that do several remote calls into other databases are needed.
</p>
</li>
<li>
<p>
Horizontal partitioning.   Using hashtext function any field can be
  converted into integer. In simpler case you can use just your id field.
  Number of partitions must be power of two in cluster and PL/Proxy uses
  bitwise and to get number of partition from given integer.
</p>
</li>
<li>
<p>
Two-level vertical partitioning.  PL/Proxy allows the cluster name also
  be calculated on function arguments.  So it is possible to dedicate
  different clusters to different categories or one cluster to read-queries,
  second cluster to write-queries and then do the usual hash-based
  partitioning inside clusters.
</p>
</li>
<li>
<p>
Read only replicas.  Load can be divided on read only replicas.  You can define
  cluster to have more partitions in cluster that you have actual databases and
  use repeating connect strings as weights on servers.
</p>
</li>
</ul></div>
<div class="paragraph"><p>In many of these scenarios good replication software like Londiste from SkyTools
is handy.</p></div>
</div>
<div class="sect2">
<h3 id="_how_to_spread_single_large_query_over_several_partitions">How to spread single large query over several partitions?</h3>
<div class="paragraph"><p>If each partition holds only part of the total data this
happens automatically - just use RUN ON ALL.</p></div>
<div class="paragraph"><p>If the partitions are copies of each other or the query does
not follow the split pattern for some other reason, it will
be bit more tricky.  Best way would be to assign each partition
number and later pass an array of parameters to RUN ON ALL;
query where each partition picks it&#8217;s values to work on.
The values can be even actual SQL queries, giving maximum
flexibility on whats possible to do.</p></div>
<div class="paragraph"><p>There is a preliminary design for feature that lets user
run a query on different partitions with different parameters:
<a href="http://lists.pgfoundry.org/pipermail/plproxy-users/2008-June/000093.html">http://lists.pgfoundry.org/pipermail/plproxy-users/2008-June/000093.html</a></p></div>
<div class="paragraph"><p>Whether this gets implemented depends if there is any <strong>actual</strong>
use-cases for this and whether there are any developers interested
in working on the feature.</p></div>
</div>
<div class="sect2">
<h3 id="_how_to_do_aggregated_queries">How to do aggregated queries?</h3>
<div class="paragraph"><p>Aggregation needs to happen in 3 steps:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Function on partition that does per-partition aggregation.
</p>
</li>
<li>
<p>
PL/Proxy function that collects the result of per-partition aggregation.
   It will return a row for each partition.
</p>
</li>
<li>
<p>
Top-level aggregation that does the final aggregation on the
   resultset of PL/Proxy function.  A regular PL/pgSQL function
   can be used or this can be done outside database by client application.
</p>
</li>
</ol></div>
<div class="paragraph"><p>Note: some of the aggregations cannot be done naively - eg. <code>avg()</code>.
Instead each partition must do <code>sum() + count()</code> and the top-level
aggregator calculates actual average.</p></div>
</div>
<div class="sect2">
<h3 id="_how_to_add_partitions">How to add partitions?</h3>
<div class="paragraph"><p>The simple way would be to collect data from all partitions
together then split it again to new partitions.  But that
is a waste of resources.</p></div>
<div class="paragraph"><p>Few things to keep in mind to make the addition easier:</p></div>
<div class="ulist"><ul>
<li>
<p>
Always partition data to more pieces that you actually need.
  Eg. if you think 2 servers would handle the load, then
  do the split into 8 partitions, keeping 4 of them
  on single server.  That way when load grows you just
  need to move databases to separate server, not rehash
  your data.  That also allows you to load-balance between
  servers with inequal power - keep more partitions on server
  that has more power.
</p>
</li>
<li>
<p>
Split one partition at a time, splitting it to 2 (preferably 4 or 8).
  You just need to keep duplicate entries in partition list
  for partitions that are not split yet.
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_can_i_have_foreign_keys_on_my_data">Can I have foreign keys on my data?</h3>
<div class="paragraph"><p>Yes, unless the data you want to partition on references
itself.</p></div>
<div class="paragraph"><p>Another common scenario is that there are some big data
tables that user wants to partition but they reference
various smaller common tables that are not partitionable.
In such situation the common tables should be managed
from single external database and replicated to each
partition.  That gives single place to manipulate data
and correct transactionality when spreading data out.</p></div>
</div>
<div class="sect2">
<h3 id="_what_happens_if_i_do_updates_in_remote_database">What happens if I do updates in remote database?</h3>
<div class="paragraph"><p>PL/Proxy is in autocommit mode so if remote function succeeds then changes are
automatically committed at once. Special handling is needed if updates are done
in both databases. If remote call fails both are rolled back but if remote call
succeeds and local updates fail then only local updates are rolled back.
Usually PgQ based solutions are used in these situations.</p></div>
</div>
<div class="sect2">
<h3 id="_how_to_handle_sequences">How to handle sequences?</h3>
<div class="paragraph"><p>Best way is to use separate ranges for each partition.</p></div>
<div class="paragraph"><p>In our case, no code uses serials directly, instead they
use wrapper function that combines unique ID each database
has and plain sequence.  That way we don&#8217;t need to manage
sequences explicitly, instead only thing we need to do
is to assign each database unique ID.</p></div>
</div>
</div>
</div>
</div>
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
Last updated 2012-11-27 14:04:52 CET
</div>
</div>
</body>
</html>
postgresql-9.4-plproxy 2.5-5 / usr / share / doc / postgresql-9.4-plproxy / faq.html
