Threadloop: Tornado IOLoop Backed Concurrent Futures

1.0.2 · maintenance · verified Thu Apr 16

Threadloop is a Python library that enables running Tornado coroutines from synchronous Python code by leveraging a thread pool, backed by Tornado's IOLoop. It provides a `ThreadLoopExecutor` that mimics the `concurrent.futures.ThreadPoolExecutor` API. The library is currently at version 1.0.2 and was last updated in 2016, indicating it is no longer actively maintained but generally functional.

Common errors

```
TypeError: 'str' object is not callable
```
cause Passing the result of a function call (e.g., `my_function()`) instead of the function object itself (`my_function`) to `executor.submit()` or `executor.map()`. This means the function is executed immediately, and its return value (often a non-callable) is then passed to `submit`.

fix Pass the function object without parentheses: `executor.submit(my_function, arg1, arg2)` instead of `executor.submit(my_function(arg1, arg2))`. Arguments should be passed separately to `submit` or within the iterable for `map`.
```
RuntimeError: cannot join current thread
```
cause Attempting to call `join()` on the current thread, which would cause a deadlock, or attempting to `join()` a thread before it has been started. While `ThreadLoopExecutor` manages its own joining, incorrect manual interaction with underlying `Thread` objects can lead to this.

fix Ensure you are not attempting to `join()` a thread from within that same thread. When using `ThreadLoopExecutor`, rely on its context manager for proper shutdown (`with ... as executor:`) which handles joining implicitly. If managing `Thread` objects directly, ensure `thread.start()` is called before `thread.join()`.

Warnings

breaking The `threadloop` library has not been updated since 2016 (version 1.0.2) and may have compatibility issues with newer Python (3.6+) and Tornado versions. Newer Python versions, especially 3.13+, introduce significant changes to threading (e.g., free-threaded CPython without GIL by default), which might affect `threadloop`'s behavior or performance.
Fix: Thoroughly test `threadloop` with your specific Python and Tornado versions. Consider modern alternatives like `concurrent.futures.ThreadPoolExecutor` for general threading or `asyncio` with `loop.run_in_executor` for integrating blocking calls into an event loop for new projects.
gotcha Tasks submitted to `ThreadLoopExecutor` (or any `concurrent.futures.ThreadPoolExecutor`) can fail silently if their `Future` objects are not inspected (e.g., by calling `.result()`, `.exception()`, or attaching `add_done_callback`). Exceptions raised within submitted tasks will not propagate to the main thread unless explicitly retrieved.
Fix: Always retrieve the result or check for exceptions on `Future` objects returned by `submit()` or `map()`. For example, `future.result()` will re-raise any exception that occurred in the worker thread. Alternatively, use `future.add_done_callback(error_handler)` to process results or errors asynchronously.
gotcha Python's Global Interpreter Lock (GIL) means that CPU-bound tasks executed within `ThreadLoopExecutor` (which uses threads) will not achieve true parallel execution across multiple CPU cores in standard CPython interpreters (pre-3.13 free-threaded builds). Threads are best for I/O-bound tasks where the GIL is released during blocking operations.
Fix: For CPU-bound parallelism, use `multiprocessing.Pool` or `concurrent.futures.ProcessPoolExecutor` which utilize separate processes, bypassing the GIL. For I/O-bound tasks, threading is generally effective. Be aware that Python 3.13+ with free-threaded builds can offer true parallelism for CPU-bound threaded code.
gotcha Attempting to submit tasks to a `ThreadLoopExecutor` after it has been shut down will result in a `RuntimeError` or similar exception. This can happen if the executor's lifecycle is not managed correctly, especially when not using it as a context manager.
Fix: Always use `ThreadLoopExecutor` as a context manager (`with ThreadLoopExecutor() as executor:`) to ensure proper shutdown and resource management. If manually managing, ensure `executor.shutdown()` is called only after all tasks have been submitted and you are ready to terminate the pool.

Install

pip install threadloop Install stable release

Imports

ThreadLoopExecutor

from threadloop import ThreadLoopExecutor

Quickstart

This quickstart demonstrates how to initialize `ThreadLoopExecutor` as a context manager and submit both Tornado coroutines and regular synchronous functions. The `submit` method returns a future, and `.result()` can be called to block until the task completes and retrieve its return value. Note that `tornado.gen.coroutine` is used for demonstration, which is an older decorator; modern Tornado often uses `async def` functions.

import tornado.gen
from threadloop import ThreadLoopExecutor

@tornado.gen.coroutine
def my_coroutine(name):
    print(f"Hello from {name} in coroutine!")
    yield tornado.gen.sleep(0.1) # Simulate async work
    print(f"Goodbye from {name}!")
    return f"Result from {name}"

def sync_function(x, y):
    print(f"Running sync_function with {x} and {y}")
    return x + y

if __name__ == '__main__':
    # Example 1: Running a Tornado coroutine in the ThreadLoopExecutor
    with ThreadLoopExecutor() as executor:
        future_coro = executor.submit(my_coroutine, "CoroutineWorker")
        result_coro = future_coro.result() # Blocks until coroutine completes
        print(f"Coroutine result: {result_coro}")

    # Example 2: Running a regular synchronous function
    with ThreadLoopExecutor() as executor:
        future_sync = executor.submit(sync_function, 10, 20)
        result_sync = future_sync.result()
        print(f"Sync function result: {result_sync}")

view raw JSON →