Export Oassis batch results to Google Cloud Storage

Name: Oassis
Price: 0.002 USD
Author: Oassis

Your application retrieves the completed batch and writes it to your bucket. Oassis does not currently offer a documented native GCS push destination.

Architecture and permissions

Start a batch → persist its jobId → poll with the owning Oassis key → inspect the result → upload JSON to GCS. Keep both credentials on your runner, not in a browser or page URL.

Use Google Application Default Credentials or workload identity for the runner. Grant its identity roles/storage.objectCreator on the destination bucket for new objects; this example uses unique names and refuses overwrite. Keep the bucket private. Google Cloud storage and operation charges are separate.

Terraform: grant access to an existing bucket

This grants create-only access to an existing runner service account. Review the variables before applying it. It does not create a public bucket or store credentials.

variable "bucket_name" { type = string }
variable "runner_service_account_email" { type = string }
resource "google_storage_bucket_iam_member" "oassis_export_writer" {
  bucket = var.bucket_name
  role   = "roles/storage.objectCreator"
  member = "serviceAccount:${var.runner_service_account_email}"
}

Python collector

Install requests and google-cloud-storage. Set OASSIS_API_KEY, OASSIS_JOB_ID and GCS_BUCKET in your runner. Run this collector periodically until the job reaches a terminal state; it does not create another paid job.

import json, os, uuid
import requests
from google.cloud import storage

job_id = os.environ['OASSIS_JOB_ID']
response = requests.get(
    'https://api.oassis.dev/tasks/v1/scrape/batch/' + job_id,
    headers={'Authorization': 'Bearer ' + os.environ['OASSIS_API_KEY']},
    timeout=60,
)
response.raise_for_status()
result = response.json()
if result.get('status') not in ('done', 'cancelled'):
    raise SystemExit('Job still running; poll later with backoff')
# Preserve per-page errors too; done does not imply every page succeeded.
name = 'oassis/' + job_id + '/' + str(uuid.uuid4()) + '.json'
blob = storage.Client().bucket(os.environ['GCS_BUCKET']).blob(name)
blob.upload_from_string(json.dumps(result), content_type='application/json',
                        if_generation_match=0)
print('Saved ' + name)

Check the current job response schema and your runner's retry behavior. Record the uploaded object name in your job store to avoid duplicate exports. Never send your GCS credentials to Oassis.

Google documentation

Upload objects · Application Default Credentials · Cloud Storage IAM roles

Try Scrape & Map · API quick start · Estimate your cost

Oassis basic fresh Markdown extraction: $0.002 per page output. All operation prices and calculator.