A scheduled Python report should select an explicit reporting day, produce deterministic output, upload it to a private S3-compatible bucket, and exit with a meaningful status. A finite serverless run fits that lifecycle better than a web request that keeps working after its response.
This tutorial turns a small synthetic sales fixture into a daily CSV and uploads it with Boto3. The fixture makes the example runnable without customer data or an application database. For production reporting, replace the fixture with an authoritative, bounded extraction; a CSV baked into an image does not refresh itself each night.
Prerequisites and the execution model
You need Python 3.13 for the example, a private S3-compatible bucket, a narrowly scoped storage credential, and a container image or source deployment. On DanubeData, Rapids Runs is currently documented as preview access for selected accounts. Confirm access before designing a production dependency around it.
Download the source, fixture, Dockerfile and tests. A run uses the parent container’s image and resource profile, with a different command. The example image has a minimal HTTP default so its parent container can satisfy the service model; the report itself executes only when invoked as a finite command. It does not need its own public report endpoint.
Use the endpoint, region and bucket name from your storage connection details. DanubeData documents https://s3.danubedata.ro and fsn1 for its object storage, but copy the values shown for your bucket. See the S3 quick start. Keep the bucket private; do not publish a report merely to make verification easier.
1. Create a small input and pin dependencies
Save this as sales.csv. Integer cents avoid floating-point addition for this deliberately simple monetary example. It supports only non-negative amounts in one implied currency; real refunds, currency conversion and accounting rules need an explicit model rather than extending the fixture silently.
date,country,amount_cents
2026-10-31,RO,2000
2026-10-31,DE,3500
2026-10-31,RO,3000
2026-10-30,RO,9000boto3==1.40.65
tzdata==2025.2Create a virtual environment and install the dependency file. The pinned versions provide a reproducible tutorial starting point, not a claim that they are the latest release. Review updates and security advisories through your normal application maintenance process.
python3.13 -m venv .venv
.venv/bin/pip install -r requirements.txt2. Build deterministic CSV bytes and upload them
Save this complete script as report.py. It defaults to yesterday in REPORT_TZ, accepts a date for replay, sorts output countries, and stores every successful rerun under the same logical daily key. Python’s zoneinfo uses IANA timezone data; the tzdata dependency provides a fallback when the image lacks system timezone files.
import argparse
import csv
import io
import json
import os
import re
import sys
from datetime import date, datetime, timedelta
from zoneinfo import ZoneInfo
import boto3
from botocore.config import Config
def report_day(now, zone):
return now.astimezone(ZoneInfo(zone)).date() - timedelta(days=1)
def build_csv(path, day):
totals = {}
with open(path, newline='', encoding='utf-8') as source:
reader = csv.DictReader(source)
if reader.fieldnames != ['date', 'country', 'amount_cents']:
raise ValueError('Unexpected input columns')
for row in reader:
parsed = date.fromisoformat(row['date'])
if not re.fullmatch('[A-Z]{2}', row['country'] or ''):
raise ValueError('Invalid country')
if not re.fullmatch(r'\d{1,12}', row['amount_cents'] or ''):
raise ValueError('Invalid amount')
if parsed == day:
country = row['country']
totals[country] = totals.get(country, 0) + int(row['amount_cents'])
output = io.StringIO(newline='')
writer = csv.writer(output, lineterminator='\n')
writer.writerow(['date', 'country', 'total_cents'])
for country in sorted(totals):
writer.writerow([day.isoformat(), country, totals[country]])
return output.getvalue().encode('utf-8')
def s3_client():
endpoint = os.environ['S3_ENDPOINT_URL']
if not endpoint.startswith('https://'):
raise ValueError('Use an HTTPS S3 endpoint')
return boto3.client('s3', endpoint_url=endpoint,
region_name=os.environ['AWS_DEFAULT_REGION'],
config=Config(signature_version='s3v4',
s3={'addressing_style': 'path'},
connect_timeout=5, read_timeout=30,
retries={'mode': 'standard', 'total_max_attempts': 3},
request_checksum_calculation='when_required'))
def upload(client, body, day):
key = f'reports/sales/{day.isoformat()}.csv'
client.put_object(Bucket=os.environ['S3_BUCKET'], Key=key,
Body=body, ContentType='text/csv; charset=utf-8')
return key
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--date', type=date.fromisoformat)
parser.add_argument('--input', default='sales.csv')
parser.add_argument('--local-output')
args = parser.parse_args()
try:
day = args.date or report_day(datetime.now().astimezone(),
os.environ.get('REPORT_TZ', 'Europe/Bucharest'))
body = build_csv(args.input, day)
if args.local_output:
with open(args.local_output, 'wb') as output:
output.write(body)
key = None
else:
key = upload(s3_client(), body, day)
print(json.dumps({'status': 'succeeded', 'date': day.isoformat(),
'bytes': len(body), 'object_key': key}))
return 0
except Exception:
print(json.dumps({'status': 'failed', 'message': 'Report generation or upload failed'}),
file=sys.stderr)
return 1
if __name__ == '__main__':
raise SystemExit(main())Configure REPORT_TZ=Europe/Bucharest, S3_ENDPOINT_URL, AWS_DEFAULT_REGION and S3_BUCKET. Inject AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY through the container environment or your secret management workflow. Do not bake them into the image or paste their values into a schedule command. The SDK reads credentials from its standard provider chain.
The upload uses the Boto3 PutObject API with path-style addressing, an explicit HTTPS endpoint and bounded SDK retries. Check compatibility with your S3 provider and your chosen SDK version. Three total SDK attempts do not mean the platform retries a failed run. An accepted upload is a storage operation, not proof that a downstream reader consumed the report.
3. Verify the output without a storage account
.venv/bin/python report.py --date 2026-10-31 --local-output report.csv
.venv/bin/python -m unittest -v test_report.pyThe resulting CSV is:
date,country,total_cents
2026-10-31,DE,3500
2026-10-31,RO,5000The tests check exact CSV bytes, repeated generation, timezone date selection, upload parameters and an access-denied failure with Botocore Stubber. These are local checks; the storage response is simulated. Before scheduling real data, upload the fixture to a test bucket you control, read it back, and compare bytes. That step validates credentials, network access, endpoint compatibility and permissions that a stub cannot prove.
4. Package a container and run it once
FROM python:3.13-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY report.py sales.csv health.py ./
USER 10001:10001
EXPOSE 8080
CMD ["python", "health.py"]Save this health-only server as health.py. It returns a fixed response and never serves local files.
import os
from http.server import BaseHTTPRequestHandler, HTTPServer
class Health(BaseHTTPRequestHandler):
def do_GET(self):
healthy = self.path == '/healthz'
self.send_response(200 if healthy else 404)
self.end_headers()
if healthy:
self.wfile.write(b'ok')
def log_message(self, format, *args):
return
if __name__ == '__main__':
HTTPServer(('0.0.0.0', int(os.environ.get('PORT', '8080'))), Health).serve_forever()The HTTP default exposes only the health route and does not read the application source or temporary directory. For a reporting-only parent container, select scale-to-zero where appropriate; an idle HTTP service and its configuration still need their own operational review. The report is small enough to hold the generated CSV in memory; use streaming or multipart upload for large exports rather than increasing memory indefinitely.
You can deploy a local Dockerfile project using the documented DanubeData CLI workflow:
danube login
danube rapids create --name daily-reports --type zip_upload --source-type dockerfile --port 8080
danube rapids deploy daily-reports --dir ./Set the environment through the container’s configuration, wait for the image deployment to finish, and inspect its actual readiness. An accepted deployment is not a successful report. In the container’s Runs tab, start the command python report.py --date 2026-10-31 with a 900-second timeout. First use the fixture and a test destination; check status, exit code and uploaded object.
5. Schedule the same image and command
Under Runs, use New schedule, or send the following request to the documented API. Supply your container ID and an appropriately scoped API token through your existing authenticated workflow. The schedule deliberately chooses 06:00, away from the usual overnight daylight-saving transition hours.
curl -X POST "https://danubedata.ro/api/v1/serverless/$CONTAINER/schedules" \
-H "Authorization: Bearer $DANUBEDATA_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "daily-sales",
"cron_expression": "0 6 * * *",
"timezone": "Europe/Bucharest",
"command": ["python", "report.py"],
"timeout_seconds": 900,
"enabled": true
}'Keep REPORT_TZ equal to the schedule timezone. Scheduled runs inherit the container environment and schedules accept no environment overrides. A one-off run can override environment values for that invocation, but that does not change future scheduled runs. Pin an image tag if you need the schedule tied to a reviewed build rather than following the parent image.
The default reporting date is based on actual execution time. If an occurrence starts a full day late, “yesterday” may be the wrong period. For strict completeness, persist a reporting cursor and reconcile missing days, or dispatch explicit dated runs from a durable controller. Keep --date for manual backfills; a cron schedule alone is not a complete accounting ledger.
After each occurrence, inspect GET /api/v1/serverless/{container_id}/runs/{run_id} until terminal is true and require status=succeeded. Retrieve its logs at the same path plus /logs. Monitor that the expected object exists for each reporting day; a healthy scheduler heartbeat cannot prove an individual report arrived.
Failures, timeouts and safe reruns
Runs permit one queued or running invocation per container. Another manual start returns 409, and a schedule occurrence is skipped while an earlier run remains active. Skips are recorded and are not automatically replayed. Do not use the same parent container for unrelated long maintenance commands if they could block the report.
The documented timeout range is 60–3600 seconds, with 900 seconds by default. Waiting for capacity can consume that window. If a run never starts, inspect its diagnostic message and quota state rather than assuming Python failed. A non-zero exit means report generation or upload failed; a platform timeout or cancellation can interrupt the script before it writes a final log entry.
Rerun a failed day with an explicit date. The same key replaces the current object for that day, so retries do not create timestamp-named duplicate reports. Versioning may retain earlier versions and storage charges; Object Lock or a write-once policy can deliberately reject replacement. Decide which behaviour your reporting policy requires. If the source extraction changes between attempts, the regenerated CSV can change even though its key stays stable.
For access-denied errors, check the credential’s destination permissions and bucket policy. For endpoint errors, verify DNS, HTTPS and region. For empty reports, inspect input freshness and the selected date: this script produces a header-only CSV when no rows match. For retention and access planning, read S3 storage security practices and S3 versioning and data protection. Explore Rapids with S3-compatible storage when the execution limits match your report.
Frequently asked questions
Can a scheduled report run for several hours?
Not within the documented one-hour Rapids Run limit. Split it into resumable bounded units or choose another worker environment. Raising an HTTP request timeout is a separate setting and does not extend the run limit.
Does the schedule store S3 secrets?
The schedule uses its parent container environment. Configure secrets there through the approved workflow. Schedule environment overrides are not supported.
Does a repeated run create duplicate files?
It writes the same daily key. The bucket’s versioning and retention settings determine whether older versions remain. Downstream systems must still handle repeated notifications or repeated reads safely.
Is a successful run enough to audit delivery?
It proves the process exited zero under this script’s rules. A stronger check verifies the expected key, date, row count and content digest and records that the consuming system accepted it.
Sources
- docs.danubedata.ro/rapids-runs
- docs.danubedata.ro/serverless-overview
- docs.danubedata.ro/cli-serverless
- docs.danubedata.ro/rapids-automation
- docs.danubedata.ro/object-storage-quickstart
- docs.python.org/3/library/zoneinfo.html
- docs.aws.amazon.com/boto3/latest/reference/services/s3/client/put_object.html
