Originally posted on StackOverflow.

How many background threads can run in parallel in a single App Engine backend?

For the record, App Engine distinguishes between normal threads, which can’t outlive the HTTP request that started them, and background threads, which can.

For Python, at least, it looks like the production (Python 2.7) runtime and dev_appserver both impose a fixed limit of 10 background threads per backend, independent of other settings like e.g. max_concurrent_requests in backends.yaml.

I’ve talked with a few other old App Engine team members, and while they weren’t 100% sure, they said this sounded right. I’ve confirmed empirically with the test backend config and code below. I also tried starting more background threads from a separate HTTP request, and from yet another background thread. No luck; same overall limit of 10 total.

Here’s where the SDK sets that limit in dev_appserver (specifically devappserver2 in SDK 1.8.8) for Python: instance.py:449, python_runtime.py:61. It looks like background threads are disabled altogether for Go and Java: go_runtime.py:99, java_runtime.py:61.

One interesting quirk: inside a background thread, it looks like you can start as many normal threads as you want, at least until you hit the memory limit. They don’t hold any HTTP request open, and they don’t seem to get cut off by a deadline either. I’m currently doing this to work around the background thread limit.

I wish this was documented! Would have saved me a lot of time.

backends.yaml

- name: test
  instances: 1
  start: threadtest.application

test.py

def test():
  for i in range(100):
    logging.info('Starting #%d', i)
    background_thread.start_new_background_thread(time.sleep, [20])

class Start(webapp2.RequestHandler):
  def get(self):
    background_thread.start_new_background_thread(test, [])

application = webapp2.WSGIApplication([('/_ah/start', Start)], debug=True)