| |
| <!DOCTYPE html> |
| |
| <html> |
| <head> |
| <meta charset="utf-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" /> |
| |
| <title>Throttling: ThrottleExecutor — more-executors documentation</title> |
| <link rel="stylesheet" type="text/css" href="../_static/pygments.css" /> |
| <link rel="stylesheet" type="text/css" href="../_static/alabaster.css" /> |
| <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script> |
| <script src="../_static/jquery.js"></script> |
| <script src="../_static/underscore.js"></script> |
| <script src="../_static/doctools.js"></script> |
| <link rel="index" title="Index" href="../genindex.html" /> |
| <link rel="search" title="Search" href="../search.html" /> |
| <link rel="next" title="Timing out futures: f_timeout, TimeoutExecutor" href="timeout.html" /> |
| <link rel="prev" title="Retrying: RetryExecutor" href="retry.html" /> |
| |
| <link rel="stylesheet" href="../_static/custom.css" type="text/css" /> |
| |
| |
| <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" /> |
| |
| </head><body> |
| |
| |
| <div class="document"> |
| <div class="documentwrapper"> |
| <div class="bodywrapper"> |
| |
| |
| <div class="body" role="main"> |
| |
| <section id="throttling-throttleexecutor"> |
| <h1>Throttling: <code class="docutils literal notranslate"><span class="pre">ThrottleExecutor</span></code><a class="headerlink" href="#throttling-throttleexecutor" title="Permalink to this headline">¶</a></h1> |
| <p>ThrottleExecutor limits the number of concurrently executing futures.</p> |
| <dl class="py class"> |
| <dt class="sig sig-object py" id="more_executors.ThrottleExecutor"> |
| <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">more_executors.</span></span><span class="sig-name descname"><span class="pre">ThrottleExecutor</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">delegate</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">count</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">logger</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'default'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">block</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#more_executors.ThrottleExecutor" title="Permalink to this definition">¶</a></dt> |
| <dd><p>An executor which delegates to another executor while enforcing |
| a limit on the number of futures running concurrently.</p> |
| <ul class="simple"> |
| <li><p>Callables are submitted to the delegate executor, from a different |
| thread than the calling thread.</p></li> |
| <li><p>Where <cite>count</cite> is used to initialize this executor, if there |
| are already <cite>count</cite> futures submitted to the delegate executor and not |
| yet <a class="reference external" href="https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future.done" title="(in Python v3.11)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">done()</span></code></a>, additional callables will |
| either be queued or will block on submit, and will only be submitted |
| to the delegate executor once there are less than <cite>count</cite> futures in |
| progress.</p></li> |
| </ul> |
| <div class="versionadded"> |
| <p><span class="versionmodified added">New in version 1.9.0.</span></p> |
| </div> |
| <dl class="field-list simple"> |
| <dt class="field-odd">Parameters</dt> |
| <dd class="field-odd"><ul class="simple"> |
| <li><p><strong>delegate</strong> (<a class="reference external" href="https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor" title="(in Python v3.11)"><em>Executor</em></a>) – an executor to which callables will be submitted</p></li> |
| <li><p><strong>count</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em>, </em><em>callable</em>) – <dl> |
| <dt>int:</dt><dd><p>maximum number of concurrently running futures.</p> |
| </dd> |
| <dt>callable:</dt><dd><p>a callable which returns an <code class="docutils literal notranslate"><span class="pre">int</span></code> (or <code class="docutils literal notranslate"><span class="pre">None</span></code>, to indicate |
| no throttling).</p> |
| <p>The callable will be invoked each time this executor needs |
| to decide whether to throttle futures; this may be used |
| to implement dynamic throttling.</p> |
| <div class="versionadded"> |
| <p><span class="versionmodified added">New in version 2.5.0.</span></p> |
| </div> |
| </dd> |
| </dl> |
| </p></li> |
| <li><p><strong>block</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – <p>If <code class="docutils literal notranslate"><span class="pre">True</span></code>, calls to <code class="docutils literal notranslate"><span class="pre">submit()</span></code> on this executor may block if |
| there are already <code class="docutils literal notranslate"><span class="pre">count</span></code> futures in progress.</p> |
| <p>Otherwise, calls to <code class="docutils literal notranslate"><span class="pre">submit()</span></code> will always return immediately |
| and callables will be queued internally.</p> |
| </p></li> |
| <li><p><strong>logger</strong> (<a class="reference external" href="https://docs.python.org/3/library/logging.html#logging.Logger" title="(in Python v3.11)"><em>Logger</em></a>) – a logger used for messages from this executor</p></li> |
| <li><p><strong>name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a name for this executor</p></li> |
| </ul> |
| </dd> |
| </dl> |
| <div class="versionchanged"> |
| <p><span class="versionmodified changed">Changed in version 2.7.0: </span>Introduced <code class="docutils literal notranslate"><span class="pre">name</span></code>.</p> |
| </div> |
| <div class="versionchanged"> |
| <p><span class="versionmodified changed">Changed in version 2.11.0: </span>Introduced <code class="docutils literal notranslate"><span class="pre">block</span></code>.</p> |
| </div> |
| </dd></dl> |
| |
| <dl class="py method"> |
| <dt class="sig sig-object py" id="more_executors.Executors.with_throttle"> |
| <em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">Executors.</span></span><span class="sig-name descname"><span class="pre">with_throttle</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">executor</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">count</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">logger</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">block</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#more_executors.Executors.with_throttle" title="Permalink to this definition">¶</a></dt> |
| <dd><dl class="field-list simple"> |
| <dt class="field-odd">Returns</dt> |
| <dd class="field-odd"><p>a new executor which enforces a limit on the number of concurrently |
| pending futures.</p> |
| </dd> |
| <dt class="field-even">Return type</dt> |
| <dd class="field-even"><p><a class="reference internal" href="#more_executors.ThrottleExecutor" title="more_executors.ThrottleExecutor"><em>ThrottleExecutor</em></a></p> |
| </dd> |
| </dl> |
| <div class="versionadded"> |
| <p><span class="versionmodified added">New in version 1.9.0.</span></p> |
| </div> |
| </dd></dl> |
| |
| </section> |
| |
| |
| </div> |
| |
| </div> |
| </div> |
| <div class="sphinxsidebar" role="navigation" aria-label="main navigation"> |
| <div class="sphinxsidebarwrapper"> |
| <h1 class="logo"><a href="../index.html">more-executors</a></h1> |
| |
| |
| |
| <p class="blurb">A library of composable Python executors and futures</p> |
| |
| |
| |
| |
| |
| |
| |
| |
| <h3>Navigation</h3> |
| <p class="caption" role="heading"><span class="caption-text">Contents:</span></p> |
| <ul class="current"> |
| <li class="toctree-l1"><a class="reference internal" href="../user-guide.html">User Guide</a></li> |
| <li class="toctree-l1 current"><a class="reference internal" href="../api-reference.html">Reference</a><ul class="current"> |
| <li class="toctree-l2"><a class="reference internal" href="base-executors.html">Base executors: <code class="docutils literal notranslate"><span class="pre">thread_pool</span></code>, <code class="docutils literal notranslate"><span class="pre">process_pool</span></code>, <code class="docutils literal notranslate"><span class="pre">sync</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="retry.html">Retrying: <code class="docutils literal notranslate"><span class="pre">RetryExecutor</span></code></a></li> |
| <li class="toctree-l2 current"><a class="current reference internal" href="#">Throttling: <code class="docutils literal notranslate"><span class="pre">ThrottleExecutor</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="timeout.html">Timing out futures: <code class="docutils literal notranslate"><span class="pre">f_timeout</span></code>, <code class="docutils literal notranslate"><span class="pre">TimeoutExecutor</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="cancel.html">Cancellation of futures: <code class="docutils literal notranslate"><span class="pre">f_nocancel</span></code>, <code class="docutils literal notranslate"><span class="pre">CancelOnShutdownExecutor</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="poll.html">Polling: <code class="docutils literal notranslate"><span class="pre">PollExecutor</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="asyncio.html">asyncio bridge: <code class="docutils literal notranslate"><span class="pre">AsyncioExecutor</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="bind.html">Binding callables to executors: <code class="docutils literal notranslate"><span class="pre">bind</span></code>, <code class="docutils literal notranslate"><span class="pre">flat_bind</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="futures-create.html">Creating futures from values: <code class="docutils literal notranslate"><span class="pre">f_return*</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="futures-map.html">Transforming futures: <code class="docutils literal notranslate"><span class="pre">f_map</span></code>, <code class="docutils literal notranslate"><span class="pre">f_flat_map</span></code>, <code class="docutils literal notranslate"><span class="pre">MapExecutor</span></code>, <code class="docutils literal notranslate"><span class="pre">FlatMapExecutor</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="futures-proxy.html">Proxy futures: <code class="docutils literal notranslate"><span class="pre">f_proxy</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="futures-apply.html">Applying functions: <code class="docutils literal notranslate"><span class="pre">f_apply</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="futures-bool.html">Boolean operations: <code class="docutils literal notranslate"><span class="pre">f_or</span></code>, <code class="docutils literal notranslate"><span class="pre">f_and</span></code></a></li> |
| <li class="toctree-l2"><a class="reference internal" href="futures-lists.html">Futures and sequences: <code class="docutils literal notranslate"><span class="pre">f_traverse</span></code>, <code class="docutils literal notranslate"><span class="pre">f_sequence</span></code>, <code class="docutils literal notranslate"><span class="pre">f_zip</span></code></a></li> |
| </ul> |
| </li> |
| </ul> |
| |
| |
| <hr /> |
| <ul> |
| |
| <li class="toctree-l1"><a href="https://github.com/rohanpm/more-executors">Source</a></li> |
| |
| <li class="toctree-l1"><a href="https://pypi.python.org/pypi/more-executors">PyPI</a></li> |
| |
| </ul> |
| <div class="relations"> |
| <h3>Related Topics</h3> |
| <ul> |
| <li><a href="../index.html">Documentation overview</a><ul> |
| <li><a href="../api-reference.html">Reference</a><ul> |
| <li>Previous: <a href="retry.html" title="previous chapter">Retrying: <code class="docutils literal notranslate"><span class="pre">RetryExecutor</span></code></a></li> |
| <li>Next: <a href="timeout.html" title="next chapter">Timing out futures: <code class="docutils literal notranslate"><span class="pre">f_timeout</span></code>, <code class="docutils literal notranslate"><span class="pre">TimeoutExecutor</span></code></a></li> |
| </ul></li> |
| </ul></li> |
| </ul> |
| </div> |
| <div id="searchbox" style="display: none" role="search"> |
| <h3 id="searchlabel">Quick search</h3> |
| <div class="searchformwrapper"> |
| <form class="search" action="../search.html" method="get"> |
| <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/> |
| <input type="submit" value="Go" /> |
| </form> |
| </div> |
| </div> |
| <script>$('#searchbox').show(0);</script> |
| |
| |
| |
| |
| |
| |
| |
| |
| </div> |
| </div> |
| <div class="clearer"></div> |
| </div> |
| <div class="footer"> |
| ©2021, Rohan McGovern. |
| |
| | |
| <a href="../_sources/reference/throttle.rst.txt" |
| rel="nofollow">Page source</a> |
| </div> |
| |
| |
| <a href="https://github.com/rohanpm/more-executors" class="github"> |
| <img style="position: absolute; top: 0; right: 0; border: 0;" src="https://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" alt="Fork me on GitHub" class="github"/> |
| </a> |
| |
| |
| |
| </body> |
| </html> |
