/usr/share/doc/python-pymongo-doc/html/examples/aggregation.html is in python-pymongo-doc 2.6.3-1build1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Aggregation Examples — PyMongo 2.6.3 documentation</title>
<link rel="stylesheet" href="../_static/default.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '2.6.3',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/sidebar.js"></script>
<link rel="top" title="PyMongo 2.6.3 documentation" href="../index.html" />
<link rel="up" title="Examples" href="index.html" />
<link rel="next" title="Authentication Examples" href="authentication.html" />
<link rel="prev" title="Examples" href="index.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="authentication.html" title="Authentication Examples"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="index.html" title="Examples"
accesskey="P">previous</a> |</li>
<li><a href="../index.html">PyMongo 2.6.3 documentation</a> »</li>
<li><a href="index.html" accesskey="U">Examples</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="aggregation-examples">
<h1>Aggregation Examples<a class="headerlink" href="#aggregation-examples" title="Permalink to this headline">¶</a></h1>
<p>There are several methods of performing aggregations in MongoDB. These
examples cover the new aggregation framework, using map reduce and using the
group method.</p>
<div class="section" id="setup">
<h2>Setup<a class="headerlink" href="#setup" title="Permalink to this headline">¶</a></h2>
<p>To start, we’ll insert some example data which we can perform
aggregations on:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pymongo</span> <span class="kn">import</span> <span class="n">MongoClient</span>
<span class="gp">>>> </span><span class="n">db</span> <span class="o">=</span> <span class="n">MongoClient</span><span class="p">()</span><span class="o">.</span><span class="n">aggregation_example</span>
<span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">insert</span><span class="p">({</span><span class="s">"x"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s">"tags"</span><span class="p">:</span> <span class="p">[</span><span class="s">"dog"</span><span class="p">,</span> <span class="s">"cat"</span><span class="p">]})</span>
<span class="go">ObjectId('...')</span>
<span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">insert</span><span class="p">({</span><span class="s">"x"</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span> <span class="s">"tags"</span><span class="p">:</span> <span class="p">[</span><span class="s">"cat"</span><span class="p">]})</span>
<span class="go">ObjectId('...')</span>
<span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">insert</span><span class="p">({</span><span class="s">"x"</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span> <span class="s">"tags"</span><span class="p">:</span> <span class="p">[</span><span class="s">"mouse"</span><span class="p">,</span> <span class="s">"cat"</span><span class="p">,</span> <span class="s">"dog"</span><span class="p">]})</span>
<span class="go">ObjectId('...')</span>
<span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">insert</span><span class="p">({</span><span class="s">"x"</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s">"tags"</span><span class="p">:</span> <span class="p">[]})</span>
<span class="go">ObjectId('...')</span>
</pre></div>
</div>
</div>
<div class="section" id="aggregation-framework">
<h2>Aggregation Framework<a class="headerlink" href="#aggregation-framework" title="Permalink to this headline">¶</a></h2>
<p>This example shows how to use the
<a class="reference internal" href="../api/pymongo/collection.html#pymongo.collection.Collection.aggregate" title="pymongo.collection.Collection.aggregate"><tt class="xref py py-meth docutils literal"><span class="pre">aggregate()</span></tt></a> method to use the aggregation
framework. We’ll perform a simple aggregation to count the number of
occurrences for each tag in the <tt class="docutils literal"><span class="pre">tags</span></tt> array, across the entire collection.
To achieve this we need to pass in three operations to the pipeline.
First, we need to unwind the <tt class="docutils literal"><span class="pre">tags</span></tt> array, then group by the tags and
sum them up, finally we sort by count.</p>
<p>As python dictionaries don’t maintain order you should use <a class="reference internal" href="../api/bson/son.html#bson.son.SON" title="bson.son.SON"><tt class="xref py py-class docutils literal"><span class="pre">SON</span></tt></a>
or <tt class="xref py py-class docutils literal"><span class="pre">collections.OrderedDict</span></tt> where explicit ordering is required
eg “$sort”:</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">aggregate requires server version <strong>>= 2.1.0</strong>. The PyMongo
<a class="reference internal" href="../api/pymongo/collection.html#pymongo.collection.Collection.aggregate" title="pymongo.collection.Collection.aggregate"><tt class="xref py py-meth docutils literal"><span class="pre">aggregate()</span></tt></a> helper requires
PyMongo version <strong>>= 2.3</strong>.</p>
</div>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">bson.son</span> <span class="kn">import</span> <span class="n">SON</span>
<span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">aggregate</span><span class="p">([</span>
<span class="gp">... </span> <span class="p">{</span><span class="s">"$unwind"</span><span class="p">:</span> <span class="s">"$tags"</span><span class="p">},</span>
<span class="gp">... </span> <span class="p">{</span><span class="s">"$group"</span><span class="p">:</span> <span class="p">{</span><span class="s">"_id"</span><span class="p">:</span> <span class="s">"$tags"</span><span class="p">,</span> <span class="s">"count"</span><span class="p">:</span> <span class="p">{</span><span class="s">"$sum"</span><span class="p">:</span> <span class="mi">1</span><span class="p">}}},</span>
<span class="gp">... </span> <span class="p">{</span><span class="s">"$sort"</span><span class="p">:</span> <span class="n">SON</span><span class="p">([(</span><span class="s">"count"</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">),</span> <span class="p">(</span><span class="s">"_id"</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)])}</span>
<span class="gp">... </span> <span class="p">])</span>
<span class="gp">...</span>
<span class="go">{u'ok': 1.0, u'result': [{u'count': 3, u'_id': u'cat'}, {u'count': 2, u'_id': u'dog'}, {u'count': 1, u'_id': u'mouse'}]}</span>
</pre></div>
</div>
<p>As well as simple aggregations the aggregation framework provides projection
capabilities to reshape the returned data. Using projections and aggregation,
you can add computed fields, create new virtual sub-objects, and extract
sub-fields into the top-level of results.</p>
<div class="admonition seealso">
<p class="first admonition-title">See also</p>
<p class="last">The full documentation for MongoDB’s <a class="reference external" href="http://docs.mongodb.org/manual/applications/aggregation">aggregation framework</a></p>
</div>
</div>
<div class="section" id="map-reduce">
<h2>Map/Reduce<a class="headerlink" href="#map-reduce" title="Permalink to this headline">¶</a></h2>
<p>Another option for aggregation is to use the map reduce framework. Here we
will define <strong>map</strong> and <strong>reduce</strong> functions to also count he number of
occurrences for each tag in the <tt class="docutils literal"><span class="pre">tags</span></tt> array, across the entire collection.</p>
<p>Our <strong>map</strong> function just emits a single <cite>(key, 1)</cite> pair for each tag in
the array:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">bson.code</span> <span class="kn">import</span> <span class="n">Code</span>
<span class="gp">>>> </span><span class="n">mapper</span> <span class="o">=</span> <span class="n">Code</span><span class="p">(</span><span class="s">"""</span>
<span class="gp">... </span><span class="s"> function () {</span>
<span class="gp">... </span><span class="s"> this.tags.forEach(function(z) {</span>
<span class="gp">... </span><span class="s"> emit(z, 1);</span>
<span class="gp">... </span><span class="s"> });</span>
<span class="gp">... </span><span class="s"> }</span>
<span class="gp">... </span><span class="s"> """</span><span class="p">)</span>
</pre></div>
</div>
<p>The <strong>reduce</strong> function sums over all of the emitted values for a given key:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">reducer</span> <span class="o">=</span> <span class="n">Code</span><span class="p">(</span><span class="s">"""</span>
<span class="gp">... </span><span class="s"> function (key, values) {</span>
<span class="gp">... </span><span class="s"> var total = 0;</span>
<span class="gp">... </span><span class="s"> for (var i = 0; i < values.length; i++) {</span>
<span class="gp">... </span><span class="s"> total += values[i];</span>
<span class="gp">... </span><span class="s"> }</span>
<span class="gp">... </span><span class="s"> return total;</span>
<span class="gp">... </span><span class="s"> }</span>
<span class="gp">... </span><span class="s"> """</span><span class="p">)</span>
</pre></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">We can’t just return <tt class="docutils literal"><span class="pre">values.length</span></tt> as the <strong>reduce</strong> function
might be called iteratively on the results of other reduce steps.</p>
</div>
<p>Finally, we call <a class="reference internal" href="../api/pymongo/collection.html#pymongo.collection.Collection.map_reduce" title="pymongo.collection.Collection.map_reduce"><tt class="xref py py-meth docutils literal"><span class="pre">map_reduce()</span></tt></a> and
iterate over the result collection:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">map_reduce</span><span class="p">(</span><span class="n">mapper</span><span class="p">,</span> <span class="n">reducer</span><span class="p">,</span> <span class="s">"myresults"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">doc</span> <span class="ow">in</span> <span class="n">result</span><span class="o">.</span><span class="n">find</span><span class="p">():</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">doc</span>
<span class="gp">...</span>
<span class="go">{u'_id': u'cat', u'value': 3.0}</span>
<span class="go">{u'_id': u'dog', u'value': 2.0}</span>
<span class="go">{u'_id': u'mouse', u'value': 1.0}</span>
</pre></div>
</div>
</div>
<div class="section" id="advanced-map-reduce">
<h2>Advanced Map/Reduce<a class="headerlink" href="#advanced-map-reduce" title="Permalink to this headline">¶</a></h2>
<p>PyMongo’s API supports all of the features of MongoDB’s map/reduce engine.
One interesting feature is the ability to get more detailed results when
desired, by passing <cite>full_response=True</cite> to
<a class="reference internal" href="../api/pymongo/collection.html#pymongo.collection.Collection.map_reduce" title="pymongo.collection.Collection.map_reduce"><tt class="xref py py-meth docutils literal"><span class="pre">map_reduce()</span></tt></a>. This returns the full
response to the map/reduce command, rather than just the result collection:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">map_reduce</span><span class="p">(</span><span class="n">mapper</span><span class="p">,</span> <span class="n">reducer</span><span class="p">,</span> <span class="s">"myresults"</span><span class="p">,</span> <span class="n">full_response</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="go">{u'counts': {u'input': 4, u'reduce': 2, u'emit': 6, u'output': 3}, u'timeMillis': ..., u'ok': ..., u'result': u'...'}</span>
</pre></div>
</div>
<p>All of the optional map/reduce parameters are also supported, simply pass them
as keyword arguments. In this example we use the <cite>query</cite> parameter to limit the
documents that will be mapped over:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">map_reduce</span><span class="p">(</span><span class="n">mapper</span><span class="p">,</span> <span class="n">reducer</span><span class="p">,</span> <span class="s">"myresults"</span><span class="p">,</span> <span class="n">query</span><span class="o">=</span><span class="p">{</span><span class="s">"x"</span><span class="p">:</span> <span class="p">{</span><span class="s">"$lt"</span><span class="p">:</span> <span class="mi">2</span><span class="p">}})</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">doc</span> <span class="ow">in</span> <span class="n">result</span><span class="o">.</span><span class="n">find</span><span class="p">():</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">doc</span>
<span class="gp">...</span>
<span class="go">{u'_id': u'cat', u'value': 1.0}</span>
<span class="go">{u'_id': u'dog', u'value': 1.0}</span>
</pre></div>
</div>
<p>With MongoDB 1.8.0 or newer you can use <a class="reference internal" href="../api/bson/son.html#bson.son.SON" title="bson.son.SON"><tt class="xref py py-class docutils literal"><span class="pre">SON</span></tt></a> or
<tt class="xref py py-class docutils literal"><span class="pre">collections.OrderedDict</span></tt> to specify a different database to store the
result collection:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">bson.son</span> <span class="kn">import</span> <span class="n">SON</span>
<span class="gp">>>> </span><span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">map_reduce</span><span class="p">(</span><span class="n">mapper</span><span class="p">,</span> <span class="n">reducer</span><span class="p">,</span> <span class="n">out</span><span class="o">=</span><span class="n">SON</span><span class="p">([(</span><span class="s">"replace"</span><span class="p">,</span> <span class="s">"results"</span><span class="p">),</span> <span class="p">(</span><span class="s">"db"</span><span class="p">,</span> <span class="s">"outdb"</span><span class="p">)]),</span> <span class="n">full_response</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="go">{u'counts': {u'input': 4, u'reduce': 2, u'emit': 6, u'output': 3}, u'timeMillis': ..., u'ok': ..., u'result': {u'db': ..., u'collection': ...}}</span>
</pre></div>
</div>
<div class="admonition seealso">
<p class="first admonition-title">See also</p>
<p class="last">The full list of options for MongoDB’s <a class="reference external" href="http://www.mongodb.org/display/DOCS/MapReduce">map reduce engine</a></p>
</div>
</div>
<div class="section" id="group">
<h2>Group<a class="headerlink" href="#group" title="Permalink to this headline">¶</a></h2>
<p>The <a class="reference internal" href="../api/pymongo/collection.html#pymongo.collection.Collection.group" title="pymongo.collection.Collection.group"><tt class="xref py py-meth docutils literal"><span class="pre">group()</span></tt></a> method provides some of the
same functionality as SQL’s GROUP BY. Simpler than a map reduce you need to
provide a key to group by, an initial value for the aggregation and a
reduce function.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Doesn’t work with sharded MongoDB configurations, use aggregation or
map/reduce instead of group().</p>
</div>
<p>Here we are doing a simple group and count of the occurrences <tt class="docutils literal"><span class="pre">x</span></tt> values:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">reducer</span> <span class="o">=</span> <span class="n">Code</span><span class="p">(</span><span class="s">"""</span>
<span class="gp">... </span><span class="s"> function(obj, prev){</span>
<span class="gp">... </span><span class="s"> prev.count++;</span>
<span class="gp">... </span><span class="s"> }</span>
<span class="gp">... </span><span class="s"> """</span><span class="p">)</span>
<span class="gp">...</span>
<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">bson.son</span> <span class="kn">import</span> <span class="n">SON</span>
<span class="gp">>>> </span><span class="n">results</span> <span class="o">=</span> <span class="n">db</span><span class="o">.</span><span class="n">things</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="n">key</span><span class="o">=</span><span class="p">{</span><span class="s">"x"</span><span class="p">:</span><span class="mi">1</span><span class="p">},</span> <span class="n">condition</span><span class="o">=</span><span class="p">{},</span> <span class="n">initial</span><span class="o">=</span><span class="p">{</span><span class="s">"count"</span><span class="p">:</span> <span class="mi">0</span><span class="p">},</span> <span class="nb">reduce</span><span class="o">=</span><span class="n">reducer</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">doc</span> <span class="ow">in</span> <span class="n">results</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">doc</span>
<span class="go">{u'count': 1.0, u'x': 1.0}</span>
<span class="go">{u'count': 2.0, u'x': 2.0}</span>
<span class="go">{u'count': 1.0, u'x': 3.0}</span>
</pre></div>
</div>
<div class="admonition seealso">
<p class="first admonition-title">See also</p>
<p class="last">The full list of options for MongoDB’s <a class="reference external" href="http://www.mongodb.org/display/DOCS/Aggregation#Aggregation-Group">group method</a></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Aggregation Examples</a><ul>
<li><a class="reference internal" href="#setup">Setup</a></li>
<li><a class="reference internal" href="#aggregation-framework">Aggregation Framework</a></li>
<li><a class="reference internal" href="#map-reduce">Map/Reduce</a></li>
<li><a class="reference internal" href="#advanced-map-reduce">Advanced Map/Reduce</a></li>
<li><a class="reference internal" href="#group">Group</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="index.html"
title="previous chapter">Examples</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="authentication.html"
title="next chapter">Authentication Examples</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/examples/aggregation.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="authentication.html" title="Authentication Examples"
>next</a> |</li>
<li class="right" >
<a href="index.html" title="Examples"
>previous</a> |</li>
<li><a href="../index.html">PyMongo 2.6.3 documentation</a> »</li>
<li><a href="index.html" >Examples</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2008 - 2012, 10gen, Inc..
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.2.
</div>
</body>
</html>
|