Skip to main content
TensorPool Object Storage is S3-compatible storage for TensorPool Organizations. It works with standard S3 tools like boto3 and rclone. For high-performance storage volumes that mount directly to clusters, see Storage Volumes.

​
Key Features

  • S3 API compatible: Works with any S3-compatible tools and libraries (boto3, rclone, etc.)
  • No ingress/egress fees: Transfer data in and out without additional costs
  • Globally replicated: Data is automatically cached across all TensorPool cluster regions globally with strong consistency
    • 99.99% of objects are globally available in all regions within 15 minutes. Small files are globally available in milliseconds.
    • You always get best-case latency as if you were using the closest region, regardless of your client location.
  • Unlimited storage: Billed on usage with no size limit. See pricing for details.
  • Organization-scoped: Credentials and buckets are shared across your organization

​
Quick Start

# 1. Enable object storage for your organization
tp object-storage enable

# 2. Get your S3 credentials
tp object-storage credentials

# 3. Create a bucket
tp object-storage bucket create my-datasets

# 4. Configure your S3 client
tp object-storage configure rclone

# 5. Use any S3-compatible tool to upload data
rclone copy ./data tp:datasets/ --ignore-checksum
The --ignore-checksum flag is required for all rclone commands. TensorPool Object Storage uses a non-standard checksum format that rclone cannot verify, causing operations to fail without this flag.Setting RCLONE_IGNORE_CHECKSUM=true in your shell configuration is recommended so you don’t need to pass the flag every time. See Shell Configuration Files for how to edit your shell config.

​
Using boto3

boto3 is the official AWS SDK for Python and can be used to interact with any S3-compatible storage, including TensorPool Object Storage. 
import boto3

# Get credentials from: tp object-storage credentials
s3 = boto3.client(
    "s3",
    endpoint_url="<your-endpoint>",
    aws_access_key_id="<your-access-key>",
    aws_secret_access_key="<your-secret-key>",
)

# Upload a file
s3.upload_file("model.pt", "my-bucket", "checkpoints/model.pt")

# List objects
response = s3.list_objects_v2(Bucket="my-bucket", Prefix="checkpoints/")
for obj in response.get("Contents", []):
    print(obj["Key"])

​
Bucket Management

# List all buckets
tp object-storage bucket list

# Create a bucket
tp object-storage bucket create my-bucket

# Delete an empty bucket
tp object-storage bucket delete my-bucket

​
FUSE Mounting

Object storage buckets can optionally be mounted to TensorPool clusters via FUSE. While convenient, FUSE mounts trade performance for filesystem compatibility.
Prefer S3 API tools over FUSE mounts. Using boto3, rclone, or other S3-compatible tools is strongly recommended. FUSE mounts add significant overhead and should only be used when filesystem semantics are required.
If you need a FUSE mount, the following rclone mount command is recommended: 
rclone mount tp:${BUCKET_NAME} ${MOUNT_POINT} \
    --config=${RCLONE_CONF_PATH} \
    --no-unicode-normalization \
    --vfs-fast-fingerprint \
    --cache-dir ${CACHE_DIR} \
    --vfs-cache-max-size ${CACHE_LIMIT_KB}K \
    --vfs-cache-mode full \
    --vfs-cache-max-age 24h \
    --no-modtime \
    --vfs-cache-poll-interval 0 \
    --vfs-cache-min-free-space 50G \
    --no-checksum \
    --ignore-checksum \
    --s3-disable-checksum \
    --vfs-read-ahead 32M \
    --vfs-read-chunk-size 16M \
    --vfs-read-chunk-size-limit 128M \
    --vfs-read-chunk-streams 8 \
    --buffer-size 4M \
    --transfers 128 \
    --checkers 128 \
    --copy-links \
    --max-backlog 2000000 \
    --tpslimit 0 \
    --use-mmap \
    --tpslimit-burst 0 \
    --s3-chunk-size 64M \
    --s3-upload-concurrency 16 \
    --s3-upload-cutoff 20M \
    --dir-cache-time 15m \
    --allow-non-empty \
    --attr-timeout 15m \
    --allow-other \
    --umask 0000 \
    --vfs-write-back 5s \
    --fuse-flag writeback_cache \
    --fuse-flag big_writes \
    --fuse-flag splice_move \
    --fuse-flag splice_read \
    --fuse-flag splice_write \
    --log-level INFO \
    --log-file /tmp/rclone.log
Due to the request-based nature of object storage, every file operation incurs fixed overhead regardless of file size:
  • FUSE overhead: User-space/kernel context switches per syscall
  • S3 API overhead: HTTP request/response cycle
For large files this overhead is negligible. For small files (under 100KB), the overhead dominates the operation time. For example, a simple touch file.txt translates to 3 S3 API calls (HeadObject, PutObject, ListObjectsV2) under the hood.Traditionally cheap operations like ls are time-intensive because object storage has no directory hierarchy — listing requires querying all objects with a matching prefix.
Mounted object storage buckets are not POSIX compliant. Unsupported features:
  • Hard links
  • Setting file permissions (chmod)
  • Sticky, set-user-ID (SUID), and set-group-ID (SGID) bits
  • Updating the modification timestamp (mtime)
  • Creating and using FIFOs (first-in-first-out) pipes
  • Creating and using Unix sockets
  • Obtaining exclusive file locks
  • Unlinking an open file while it is still readable
While symlinks are supported, their use is discouraged. Symlink targets may not exist across all clusters, which can cause unexpected behavior.The use of small files (under 100KB) is discouraged due to the request-based nature of object storage.Setting up Python virtual environments within an object storage bucket is not recommended due to virtual environments’ use of symlinks and large number (~1000) of small files.

​
Next Steps