In Python, running multiple synchronous functions simultaneously is usually handled through multithreading or multiprocessing, as Python's standard asyncio.gather only works with asynchronous coroutines.
The best way to "gather" synchronous functions is by using the concurrent.futures module.
1. Using ThreadPoolExecutor (For I/O tasks)
If your functions involve waiting (like API calls, reading files, or database queries), threads are the most efficient choice.
from concurrent.futures import ThreadPoolExecutor
def fetch_data(id):
# Simulate a sync I/O task
print(f"Starting task {id}")
return f"Result from {id}"
tasks = [1, 2, 3, 4]
with ThreadPoolExecutor() as executor:
# .map returns results in the same order as the input list
results = list(executor.map(fetch_data, tasks))
print(results)
2. Using ProcessPoolExecutor (For CPU tasks)
If your functions are computationally heavy (like math, image processing, or data crunching), use processes to bypass Python's Global Interpreter Lock (GIL).
from concurrent.futures import ProcessPoolExecutor
def heavy_computation(n):
return sum(i * i for i in range(n))
numbers = [10**6, 10**7, 10**8]
if __name__ == "__main__":
with ProcessPoolExecutor() as executor:
results = list(executor.map(heavy_computation, numbers))
print(results)
Comparison:
Tip: executor.submit
While .map() is simple, .submit() gives you more control if you have different functions with different arguments:
with ThreadPoolExecutor() as executor:
future1 = executor.submit(task_a, "arg1")
future2 = executor.submit(task_b, "arg2")
results = [future1.result(), future2.result()]
Integrating synchronous code into an asynchronous asyncio loop is a common pattern, especially when using libraries that don't support async/await yet (like requests or boto3).
To do this, you use the run_in_executor method. This effectively "gathers" sync functions by running them in a background thread or process and returning a "future" that asyncio can await.
Using loop.run_in_executor
This allows you to treat a blocking synchronous function as if it were a coroutine.
import asyncio
import time
import requests
def sync_task(url):
# This is a blocking call (synchronous)
print(f"Fetching {url}...")
response = requests.get(url)
return response.status_code
async def main():
loop = asyncio.get_running_loop()
urls = ["https://google.com", "https://python.org", "https://github.com"]
# We wrap the sync calls in executor tasks
# None uses the default ThreadPoolExecutor
tasks = [
loop.run_in_executor(None, sync_task, url)
for url in urls
]
# Now we can use the real asyncio.gather!
results = await asyncio.gather(*tasks)
print(f"Results: {results}")
if __name__ == "__main__":
asyncio.run(main())
Modern Alternative: asyncio.to_thread (Python 3.9+)
If you are using a newer version of Python, there is a much cleaner "high-level" way to do this without manually touching the loop or executors.
import asyncio
import time
def blocking_io():
time.sleep(2) # Simulate a 2-second sync block
return "Done"
async def main():
print("Starting gathered sync tasks...")
# asyncio.to_thread automatically handles the executor logic
results = await asyncio.gather(
asyncio.to_thread(blocking_io),
asyncio.to_thread(blocking_io),
asyncio.to_thread(blocking_io)
)
print(f"Gathered results: {results}")
asyncio.run(main())
Key Takeaways
asyncio.gather(*tasks): Only works on awaitables (coroutines or futures).
asyncio.to_thread(func, *args): The easiest way to turn a sync function into an awaitable.
Performance: Even though you are using asyncio, these functions are still running in threads behind the scenes because Python cannot magically make a blocking function non-blocking.

