I’m migrating a Python 2 App Engine Standard app to the new Python 3 Standard runtime, which is very different. One big difference is that you use Google Cloud APIs explicitly, including Stackdriver Logging. I’m using google-cloud-logging‘s App Engine Handler and configuring it to correlate logs with HTTP requests:
import google.cloud.logging logging_client = google.cloud.logging.Client() from google.cloud.logging.handlers import AppEngineHandler, setup_logging setup_logging(AppEngineHandler(logging_client, name='stdout'))
This works great!…except it hits a 256KB log entry limit and crashes somewhat regularly. Stack trace below. This happens in long-running background requests that do a fair amount of work, but when I count the total amount of logged text (including string timestamps) in requests that crash, it only averages 5-15KB or so, far smaller than 256KB. (Yes, the docs warn This approximate limit is based on internal data sizes, but it’s hard to believe that overhead is 15-50x.)
The App Engine Handler defaults to
BackgroundThreadTransport. I’ve tried lowering its
batch_size from 10 (the default) to 1, and switching to
SyncTransport, but no luck.
Any idea what I’m missing? Thank you in advance!
Traceback (most recent call last): File "/env/lib/python3.7/site-packages/google/cloud/logging/handlers/transports/background_thread.py", line 123, in _safely_commit_batch batch.commit() File "/env/lib/python3.7/site-packages/google/cloud/logging/logger.py", line 383, in commit client.logging_api.write_entries(entries, **kwargs) File "/env/lib/python3.7/site-packages/google/cloud/logging/_gapic.py", line 126, in write_entries partial_success=partial_success, File "/env/lib/python3.7/site-packages/google/cloud/logging_v2/gapic/logging_service_v2_client.py", line 477, in write_log_entries request, retry=retry, timeout=timeout, metadata=metadata File "/env/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__ return wrapped_func(*args, **kwargs) File "/env/lib/python3.7/site-packages/google/api_core/retry.py", line 277, in retry_wrapped_func on_error=on_error, File "/env/lib/python3.7/site-packages/google/api_core/retry.py", line 182, in retry_target return target() File "/env/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout return func(*args, **kwargs) File "/env/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable six.raise_from(exceptions.from_grpc_error(exc), exc) File "<string>", line 3, in raise_from google.api_core.exceptions.InvalidArgument: 400 Log entry with size 258.3K exceeds maximum size of 256.0K