Skip to contents

CDN Operations

Source: vignettes/cdn-operations.Rmd
cdn-operations.Rmd

This vignette covers CDN (Content Delivery Network) operations in artcore. The platform uses DigitalOcean Spaces, an S3-compatible object storage service, to store artwork assets.

Understanding the Bucket Structure

The platform uses four buckets, each with a specific purpose:

Bucket Access Purpose
art-vault Private Original artwork bundles (Procreate files, signatures)
art-data Private Processed assets (frame sequences, analytics data)
art-coa Private Certificates of authenticity (PDF, JPEG)
art-public Public Thumbnails for display in the app

Key Path Conventions 

art-vault/
└── uploads/{artist_uuid}/{artwork_uuid}/
    ├── canvas.procreate
    ├── metadata.json
    └── signature.png

art-data/
└── processed/{artist_uuid}/{artwork_uuid}/
    ├── frames/
    │   ├── frame_0001.png
    │   ├── frame_0002.png
    │   └── ...
    └── analytics.json

art-coa/
└── issued/{artist_uuid}/{artwork_uuid}/
    ├── certificate.pdf
    └── certificate.jpeg

art-public/
└── thumbnails/
    ├── artist/{artist_uuid}.jpeg        # Artist avatar
    └── {artist_uuid}/{artwork_uuid}.jpeg # Artwork thumbnail

Upload Workflows

Uploading to art-vault (Original Files)

Use write_art_vault() when authenticating new artwork. This stores the original, unmodified files that can never be overwritten.

library(artcore)

artist <- "88xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
artwork <- genArtworkID()

# Upload a directory containing the artwork bundle
bundle_dir <- "path/to/bundle/"
keys <- write_art_vault(artist, artwork, local_path = bundle_dir)
#> ℹ (art-vault) Upload to Prefix -> uploads/88xxx.../99xxx.../
#> ℹ art-vault Uploading 5 file(s) => uploads/88xxx.../99xxx.../
#> ✔ art-vault Vault upload complete: 5 objects

# keys contains the S3 keys that were uploaded
print(keys)
#> [1] "uploads/88xxx.../99xxx.../canvas.procreate"
#> [2] "uploads/88xxx.../99xxx.../metadata.json"
#> ...

Uploading to art-data (Processed Files)

Use write_art_data() after processing the artwork through the pipeline. This stores frame sequences, analytics data, and other derived assets.

# Upload processed frames directory
processed_dir <- "path/to/processed/output/"
write_art_data(artist, artwork, local_path = processed_dir)
#> ℹ (art-data) Upload to Prefix -> processed/88xxx.../99xxx.../
#> ✔ art-data Upload complete: 150 objects at processed/88xxx.../99xxx.../

# Or upload a single file with explicit prefix
write_art_data(
  artist, artwork,
  local_path = "analytics.json",
  prefix = paste0("processed/", artist, "/", artwork)
)

Uploading to art-public (Thumbnails)

Use write_art_public() for thumbnails displayed in the app. These are the only assets that don’t require signed URLs.

# Upload artwork thumbnail
write_art_public(artist, artwork, local_path = "thumbnail.jpeg")
#> ℹ art-public Uploading thumb => thumbnails/88xxx.../99xxx....jpeg
#> ✔ art-public thumbnails/88xxx.../99xxx....jpeg

# Upload artist avatar (no artwork UUID)
write_art_public(artist, local_path = "avatar.jpeg")
#> ✔ art-public thumbnails/artist/88xxx....jpeg

Uploading Certificates

Use write_art_coa() for certificates of authenticity.

# Upload certificate PDF
write_art_coa(artist, artwork, local_path = "certificate.pdf")
#> ℹ art-coa Uploading cert file => issued/88xxx.../99xxx.../certificate.pdf
#> ✔ write_art_coa art-coa - issued/88xxx.../99xxx.../certificate.pdf

# Upload certificate image
write_art_coa(artist, artwork, local_path = "certificate.jpeg")

Verification Patterns

Check if Object Exists

Use has_object() to verify a specific file exists:

# Check for artwork thumbnail
has_object("art-public", paste0("thumbnails/", artist, "/", artwork, ".jpeg"))
#> ℹ KEY FOUND (art-public) thumbnails/88xxx.../99xxx....jpeg
#> [1] TRUE

# Check for certificate
has_object("art-coa", paste0("issued/", artist, "/", artwork, "/certificate.pdf"))
#> ℹ KEY NOT FOUND (art-coa) issued/88xxx.../99xxx.../certificate.pdf
#> [1] FALSE

Check if Prefix Has Objects

Use has_prefix() to check if any objects exist under a path:

# Check if artwork has been processed
has_prefix("art-data", paste0("processed/", artist, "/", artwork))
#> ℹ PREFIX FOUND (art-data) processed/88xxx.../99xxx.../
#> [1] TRUE

# Check vault for original files
has_prefix("art-vault", paste0("uploads/", artist, "/", artwork))
#> [1] TRUE

List Objects Under Prefix

Use cdn_list_keys() to enumerate all objects:

# List all processed files
keys <- cdn_list_keys("art-data", paste0("processed/", artist, "/", artwork))
length(keys)
#> [1] 152

head(keys, 5)
#> [1] "processed/88xxx.../99xxx.../frames/frame_0001.png"
#> [2] "processed/88xxx.../99xxx.../frames/frame_0002.png"
#> ...

Count Objects

Use cdn_count_keys() for quick counts without listing:

cdn_count_keys(paste0("processed/", artist, "/", artwork), bucket = "art-data")
#> [1] 152

URL Generation

Public URLs

For art-public assets, construct URLs directly:

# Public thumbnail URL (no signing needed)
cdn_asset_url("art-public", paste0("thumbnails/", artist, "/", artwork, ".jpeg"),
              signed = FALSE)
#> [1] "https://sfo3.digitaloceanspaces.com/art-public/thumbnails/88xxx.../99xxx....jpeg"

Signed URLs (Private Buckets)

For private buckets, generate presigned URLs with expiration:

# Signed URL for vault asset (1 hour expiry)
cdn_asset_url("art-vault",
              paste0("uploads/", artist, "/", artwork, "/canvas.procreate"),
              signed = TRUE)
#> [1] "https://sfo3.digitaloceanspaces.com/art-vault/uploads/...?X-Amz-Signature=..."

# With existence check
cdn_asset_url("art-data",
              paste0("processed/", artist, "/", artwork, "/analytics.json"),
              signed = TRUE, check = TRUE)
# Throws error if file doesn't exist

Delete Workflows

Use cdn_soft_delete() to move artwork assets to _deleted prefixes. This preserves files for recovery while hiding them from normal operations.

# Move artwork to _deleted prefix across all buckets
cdn_soft_delete(artist, artwork, dry_run = TRUE)
#> ℹ DRY RUN => [art-vault] prefix 'uploads/88xxx.../99xxx.../'=>'uploads/88xxx.../_deleted-99xxx.../'
#> ℹ DRY RUN => [art-data] prefix 'processed/88xxx.../99xxx.../'=>'processed/88xxx.../_deleted-99xxx.../'
#> ℹ DRY RUN => [art-coa] prefix 'issued/88xxx.../99xxx.../'=>'issued/88xxx.../_deleted-99xxx.../'
#> ℹ DRY RUN => [art-public] key 'thumbnails/88xxx.../99xxx....jpeg' => 'thumbnails/88xxx.../_deleted-99xxx....jpeg'

# Execute the soft delete
cdn_soft_delete(artist, artwork)
#> ✔ art-vault Moved 5 objects to 'uploads/88xxx.../_deleted-99xxx.../'
#> ✔ art-data Moved 152 objects to 'processed/88xxx.../_deleted-99xxx.../'
#> ✔ art-coa Moved 2 objects to 'issued/88xxx.../_deleted-99xxx.../'
#> ✔ art-public Moved thumbnail to thumbnails/88xxx.../_deleted-99xxx....jpeg

The function: - Copies each object to the new _deleted-{artwork_uuid} prefix - Deletes the original objects - Handles all four buckets in one call - Warns if expected objects are missing

Hard Delete (Permanent)

Use cdn_hard_delete() to permanently remove a directory:

# Delete empty directory placeholder
cdn_hard_delete("processed/test/empty-dir", bucket = "art-data")
#> [1] "processed/test/empty-dir/"

# Delete directory with all contents (use with caution!)
cdn_hard_delete("processed/test/experiment",
                bucket = "art-data",
                recursive = TRUE)
#> [1] "processed/test/experiment/file1.txt"
#> [2] "processed/test/experiment/file2.txt"
#> ...

Warning: Hard delete is permanent. There is no recovery. Use soft delete for normal artwork removal.

Creating Directories

S3 doesn’t have real directories, but some tools expect them. Use cdn_create_directory() to create a zero-byte placeholder:

# Create directory structure before uploading
cdn_create_directory(paste0("processed/", artist, "/", artwork), bucket = "art-data")

Error Handling

All CDN operations follow fail-fast principles:

# Missing credentials
write_art_vault(artist, artwork, "bundle/")
#> Error: ART_BUCKETS_KEY_ID and/or ART_BUCKETS_KEY_SECRET are not set.

# Attempting to overwrite
write_art_vault(artist, artwork, "bundle/")
#> Error: Vault prefix already exists: uploads/88xxx.../99xxx.../

# Invalid UUID format
write_art_vault("invalid-uuid", artwork, "bundle/")
#> Error: Invalid Artist UUID

Best Practices

  1. Always validate environment first

  2. Verify uploads before proceeding

    keys <- write_art_vault(artist, artwork, bundle)
stopifnot(has_prefix("art-vault", paste0("uploads/", artist, "/", artwork)))

  3. Use dry_run for deletes

    cdn_soft_delete(artist, artwork, dry_run = TRUE)  # Preview first
cdn_soft_delete(artist, artwork)  # Then execute

  4. Prefer soft delete over hard delete

    • Allows recovery if needed
    • Creates audit trail
    • Matches database soft delete patterns

  5. Handle large uploads in chunks

    • Files over 100MB automatically use multipart upload
    • Progress is logged for each part