Durable Service Mocks

Overview

Modern software development often involves integrating with external services. A common challenge is conducting thorough testing and efficient development without the overhead, cost, or complexity of managing live service instances. While general-purpose tools like Testcontainers (which allow you to run popular databases, message brokers, and web browsers in Docker containers) effectively manage common service dependencies, they often fall short when dealing with proprietary or highly complex external services — those that are difficult, expensive, or even impossible to replicate faithfully in development and test environments. Durable Service Mocks address this challenge.

One may wonder: how do Durable Service Mocks achieve this fidelity and efficiency? The answer lies in their Rust-based architecture and minimal SQLite backing.

These mocks offer executable implementations that accurately replicate the behavior of original services. Built with Rust, which offers strong performance and reliability — and backed by a minimal SQLite database — these mocks provide a lightweight, self-contained solution for testing. This approach enables developers to test and iterate against realistic service responses, supporting accuracy and efficiency in the development lifecycle, and fostering more sustainable and cost-effective testing strategies.

It is important to note, however, that while these mocks offer high fidelity, they are not substitutes for full production services. They excel in unit and integration testing, enabling rapid, isolated validation of application logic against external service behaviors. However, for comprehensive end-to-end testing, performance benchmarking, or scenarios involving complex cross-service interactions, direct engagement with actual service instances remains essential. This distinction is a key trade-off: mocks provide speed and isolation, while real services offer ultimate fidelity and performance insights.

Supported Services

Durable Service Mocks focus on supporting services that often present challenges in testing and development environments, particularly due to their proprietary nature, complex configurations, or high operational costs. This collection currently includes high-fidelity mock implementations for a range of services, such as:

• AWS S3 – Mock S3-compatible object storage
• Proxmox – Virtual environment management API
• OpenNebula – Cloud computing platform API
• Salesforce – CRM and business automation
• Jira – Project management and issue tracking
• Discord – Communication platform API
• Asana – Work management platform
• Box – Cloud content management
• Heroku – Platform-as-a-service API
• YouTube – Video platform API

Design Principles

The effectiveness of Durable Service Mocks in solving complex testing and development challenges hinges on a set of foundational design principles. These principles are not merely guidelines; they are the core tenets that ensure our mocks deliver the reliability, accuracy, and efficiency necessary for simulating external services. By adhering to these, we empower developers with tools that genuinely enhance their workflow:

• Reliability: Mocks are engineered for consistent and predictable behavior, which is essential for reproducible testing and stable development cycles.
• Simplicity: We prioritize straightforward setup and use, minimizing complex configurations to allow developers to integrate mocks quickly and efficiently.
• Accuracy: A faithful reproduction of original service behavior and responses is a core aspect of our approach, ensuring that tests reflect real-world interactions as closely as possible.
• Efficiency: Designed for minimal resource consumption, these mocks are well-suited for integration into CI/CD pipelines and local development environments where resource overhead must be kept low.

Getting Started

Installation

To begin using Durable Service Mocks, you’ll need to obtain the executable for the specific service you wish to mock. The primary method is to build from source, which ensures you have the latest version and allows for customization.

Building from Source:

1. Prerequisites: Ensure you have Rust and Cargo installed. We recommend using Rust stable channel.

   • Explanation: This step ensures your development environment has the necessary tools to compile Rust projects.

2. Clone the Repository:

   $ git clone https://github.com/durableprogramming/durable-service-mocks.git
   $ cd durable-service-mocks
   

   • Explanation: This command downloads the source code of the Durable Service Mocks project to your local machine and navigates into the project directory.

3. Build the Mock Service: Navigate to the specific mock service directory (e.g., crates/durable-mock-s3) and build it.

   $ cargo build --release --package durable-mock-s3
   

   • Explanation: This command compiles the specified mock service (e.g., durable-mock-s3) in release mode, optimizing it for performance. The --release flag ensures an optimized, production-ready executable, and --package specifies which crate within the workspace to build. The executable will be located in target/release/durable-mock-s3.

Using Pre-built Binaries (if available, for quicker setup):

For convenience, pre-built binaries for various platforms may be available on the GitHub Releases page. Download the appropriate binary for your operating system and ensure it’s in your system’s PATH or accessible directly.

Running a Mock Service

To illustrate the straightforward nature of using these mocks, let’s consider an illustrative example with the mock AWS S3 service. Once you have the durable-mock-s3 executable available, you can start a mock S3 server using a command like this:

$ durable-mock-s3 start --port 8000 --data-dir /tmp/mock-s3-data

Tip: If you encounter an error indicating that the port is already in use, try choosing a different port number (e.g., --port 8001). This often occurs when another application is already listening on the specified port.

This command launches a mock S3 service on http://localhost:8000, storing all its state (buckets, objects, etc.) in the /tmp/mock-s3-data directory. You also may notice the service starting up and listening for requests, as shown in the output below:

INFO  durable_mock_s3::server > Starting Durable Mock S3 server on 127.0.0.1:8000
INFO  durable_mock_s3::server > Data directory: /tmp/mock-s3-data
INFO  durable_mock_s3::server > Server ready to accept connections...

Note: The exact output, including timestamps, process IDs, or specific IP addresses, may vary depending on your environment, system configuration, and the version of the mock service. These variations are normal and do not indicate an issue; the core message — that the server is starting and ready — remains consistent.

Interacting with a Mock Service

Once the mock service is running, you can interact with it using any standard AWS S3 client, configured to point to your local mock endpoint. Here’s a Python example using boto3 to create a bucket and upload a file:

import boto3
import os

# Configure the S3 client to use the local mock endpoint
s3_client = boto3.client(
    's3',
    endpoint_url='http://localhost:8000', # Pointing to our local mock S3 server
    aws_access_key_id='test',  # Dummy credentials for the mock are sufficient
    aws_secret_access_key='test', # Dummy credentials for the mock are sufficient
    region_name='us-east-1' # A region is required, but its value is arbitrary for the mock
)

> **Note:** For mock services, dummy credentials like `aws_access_key_id='test'` and `aws_secret_access_key='test'` are typically sufficient and expected. These are not real AWS credentials and should not be used for actual AWS services.

bucket_name = 'my-test-bucket'
file_name = 'hello.txt'
file_content = 'Hello from Durable Service Mocks!'

try:
    # Create a bucket
    print(f"Creating bucket: {bucket_name}")
    s3_client.create_bucket(Bucket=bucket_name)
    print(f"Bucket '{bucket_name}' created successfully.")

    # Upload a file
    print(f"Uploading file: {file_name} to bucket: {bucket_name}")
    s3_client.put_object(Bucket=bucket_name, Key=file_name, Body=file_content)
    print(f"File '{file_name}' uploaded successfully.")

    # Verify the file content
    print(f"Downloading file: {file_name} from bucket: {bucket_name}")
    response = s3_client.get_object(Bucket=bucket_name, Key=file_name)
    downloaded_content = response['Body'].read().decode('utf-8')
    print(f"Downloaded content: '{downloaded_content}'")

    assert downloaded_content == file_content
    print("Content verification successful!")

except Exception as e:
    print(f"An error occurred: {e}")
finally:
    # Clean up (optional)
    # s3_client.delete_object(Bucket=bucket_name, Key=file_name)
    # s3_client.delete_bucket(Bucket=bucket_name)
    pass

This Python script demonstrates how to:

• Configure boto3: Connects the AWS SDK to the local durable-mock-s3 instance, overriding the default AWS endpoint.
• Create an S3 bucket: Illustrates how the mock service handles bucket creation requests, simulating real S3 behavior.
• Upload a text file: Shows the process of storing an object, confirming the mock’s ability to manage data.
• Retrieve and verify content: Downloads the uploaded file and asserts its content, validating the mock’s data persistence and retrieval accuracy.

This interaction highlights how Durable Service Mocks provide a realistic environment for development and testing without requiring actual AWS credentials or incurring cloud costs.

Technical Details

To deliver on our design principles of reliability, accuracy, and efficiency, Durable Service Mocks leverage a carefully selected technical stack.

We chose Rust for its exceptional performance, memory safety, and robust concurrency features. These qualities are paramount for creating predictable and resilient testing tools, as Rust’s ownership and borrowing system, for instance, eliminates common classes of bugs at compile time, leading to more stable and trustworthy mocks. Each service mock is then packaged as a self-contained Rust crate and executable, minimizing external dependencies and simplifying integration into any development workflow.

The underlying SQLite database provides a lightweight, embedded solution for managing mock state. Unlike traditional client-server databases, SQLite operates as a file-based database, meaning it runs within the application process itself without the need for a separate server. This choice ensures that each mock remains self-contained and resource-efficient, avoiding the overhead of external database systems and making the mocks exceptionally well-suited for rapid iteration and CI/CD environments.

Furthermore, the project’s workspace architecture is a deliberate design choice. You can think of it as a monorepo specifically tailored for Rust projects — allowing multiple related crates (like durable-mock-s3, durable-mock-proxmox, etc.) to coexist and share common code within a single repository. This structure enables the development of shared utilities and consistent patterns across all mock implementations, which in turn enhances service-specific accuracy and broad protocol support (including HTTP, HTTPS, FTP, and other specialized communication methods). This modular and consistent foundation provides developers with easy integration and an accurate testing experience, free from complex setup or resource-intensive backends.

Further Exploration

Durable Service Mocks are designed to empower developers, enabling you to build and test robust applications with confidence. We encourage you to not only experiment with the existing mock services in your projects but also to consider how you might extend or even create new mocks to suit your specific needs. The ability to simulate complex external services locally democratizes development, making it accessible to a wider range of projects and teams, whether for public open-source initiatives or private internal tools. This fosters a more inclusive and efficient development ecosystem.

We invite you to explore the various mock services available and integrate them into your testing workflows. For more detailed documentation, advanced configurations, and to contribute to the project, please visit the Durable Service Mocks GitHub repository. There, you’ll find comprehensive guides and examples to help you leverage the full potential of these tools, and we warmly welcome your contributions to expand this valuable resource.

Troubleshooting

If you encounter issues, ensure that the mock service is running on the correct port and that your client is configured to connect to http://localhost:<port> (or the specified host). Check the console output of the mock service for any error messages, which often provide clues for common configuration problems. For persistent issues, consult the project’s GitHub repository for known issues or to open a new discussion.