Rawpixel Interactive Downloader

A CLI tool for downloading CC0 (public domain) images from rawpixel.com.

Overview

Rawpixel hosts thousands of free public domain images across various collections (boards and topics). This tool provides an interactive way to browse, search, and batch download CC0-licensed images.

How It Works

Cache System

The tool maintains a local SQLite database of all rawpixel boards, topics, and image metadata. This enables:

• Fast browsing - No waiting for API calls when selecting collections
• Offline search - Search all CC0 images locally
• Change detection - Only fetch new/updated content on subsequent runs
• Deduplication - Images appearing in multiple collections are stored once with references
• Scalability - Handles millions of images without memory issues

CC0 Detection

Rawpixel doesn't mark boards or topics as CC0 at the collection level. Instead, CC0 status is determined per-image via the metadata.isCC0License flag. The tool filters collections to only show those containing CC0 images.

Collections

Content is organized into two types:

• Boards [B] - Curated collections (2,395+)
• Topics [T] - Tag-based groupings (4,000+)

Both are combined and displayed alphabetically for selection.

Features

• Interactive CLI with search and filtering
• Browse both boards and topics with CC0 images
• Download specific images or entire collections
• Set limits on images per collection
• Duplicate detection (skips already downloaded files)
• Dry run mode for previewing downloads
• Descriptive or ID-only filenames
• Resumable sync (continues where left off if interrupted)
• Optional sync on startup (shows last sync time)
• Uncategorized collection for CC0 images not in curated collections
• Search by title or ID (partial matching)

File Structure

rawpixel/
  index.js     # Main interactive downloader
  cache.js     # Cache system and sync logic
  migrate.js   # Migration script (one-time use)
  cache/
    cache.db   # SQLite database

Database Schema

boards       - Board metadata (entity_id, title, changed, data)
topics       - Topic metadata (id, title, tag, data)
images       - Image data (id, title, is_cc0, data)
image_boards - Many-to-many: images <-> boards
image_topics - Many-to-many: images <-> topics
metadata     - Key-value store (last_sync_time, etc.)

API Endpoints Used

• GET /api/v1/boards - List all boards
• GET /api/v1/topics - List all topics
• GET /api/v1/search?board={id} - Images for a board
• GET /api/v1/search?tags={tag} - Images for a topic
• POST /api/v1/user/session - Get auth token
• POST /api/v1/image/download/{id}/jpeg - Download image

Rate Limiting

API calls include random delays (10-250ms) between requests to avoid overwhelming rawpixel's servers.

API Limitations

The rawpixel API has a hard limit of 100,000 images per collection (100 items/page × 1000 pages max). Collections like "Public Domain" report 1M+ total images but only the first 100k are accessible via pagination.