Mocking Google Cloud Storage (GCS) in your local Docker environment

Why Mock GCS?

When developing applications that rely on Google Cloud Storage (GCS), testing locally can be challenging due to the need for an actual cloud environment. Fortunately, we can mock GCS using fake-gcs-server and Docker. This allows us to simulate storage behavior in a local environment without needing a real Google Cloud account.

Benefits

✅ Offline development – Work without an internet connection.

✅ Faster testing – Avoid waiting for cloud responses.

✅ No cloud costs – Reduce expenses by eliminating API calls to GCS.

✅ Predictable behavior – Control the storage environment and avoid external dependencies.

Setting Up fake-gcs-server with Docker

To run a fake GCS server locally, use the following Docker Compose configuration:

services:
  local-gcs:
    image: fsouza/fake-gcs-server
    ports:
      - "5050:5050"
    command: -scheme http -port 5050 -data /storage -public-host 0.0.0.0:5050
    volumes:
      - ./apps/<YOUR-APP>/local-gcs-bucket/data:/storage

This setup will create a local GCS server running on port 5050. I needed it to run a custom port, if you don't need to specify the port, the default is 4443. The -data /storage option specifies the directory where the server will store its data (objects, buckets, etc.). The -public-host 0.0.0.0:5050 option allows you to access the server from your local machine.

You need to ensure the path exists, in your app for example apps/MY-APP/local-gcs-bucket/data I took it one step further and created a bucket in the folder apps/MY-APP/local-gcs-bucket/data/sample-bucket. I'd recommending adding a line in the gitignore to not track any files in the sample-bucket folder.

Now, start the service:

docker-compose up -d

Updating the application gcs.ts to use the local GCS server for dev environments

Here is what I did:

import crypto from "node:crypto";
import { Storage } from "@google-cloud/storage";
import { fileTypeFromBuffer } from "file-type";
import confs from "#src/confs";
import { isLocalEnvironment } from "#src/constants/common";
import logger from "#src/lib/logging";

const INTERNAL_ENDPOINT = `http://${confs.STORAGE_BUCKET_MEDIA}:5050`;
const EXTERNAL_ENDPOINT = "http://localhost:5050";
const BUCKET_NAME = isLocalEnvironment ? "sample-bucket" : confs.STORAGE_BUCKET_MEDIA;

const storage = new Storage(
  isLocalEnvironment
    ? {
        apiEndpoint: INTERNAL_ENDPOINT,
        projectId: "local-project",
        credentials: { client_email: "test@test.com", private_key: "key" },
      }
    : undefined,
);

if (isLocalEnvironment) {
  try {
    await fetch(`${INTERNAL_ENDPOINT}/_internal/config`, {
      method: "PUT",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        externalUrl: EXTERNAL_ENDPOINT,
        publicHost: "localhost:5050",
        cors: [{ origin: ["*"], method: ["*"], responseHeader: ["*"], maxAgeSeconds: 3600 }],
      }),
    });
  } catch (error) {
    logger.error("Error configuring fake-gcs-server:", error);
  }
}

For the chunk of code above, you can see why we use the sample-bucket. You may not want to or need to do this. For anything that is not a local environment, we use the STORAGE_BUCKET_MEDIA variable.

Test it out

Now in your local environment, try uploading a file and see if it's working. That is of course, assuming you already have the code written to do so...if not, you'll need to tackle that first.

If you run into issues, I'd recommend logging into the fake-gcs-container and see what's going on.

docker exec -it fake-gcs-container

Conclusion

Prior to implementing this flow, the team had two ways, which weren't great to verify the functionality they were working on related to storage buckets.

We would either hard code an image in the development environment, or we would need to verify in the review environment, meaning it had to be pushed up to github and provisioned before the team could see if things were working as expected.

Since implementing this flow, the team can now test in their local environment and know that their code is working as expected before it's even pushed up to github.