Docker Compose provides a powerful way to run n8n with custom configurations and additional NPM modules. This guide will walk you through setting up a production-ready n8n instance using Docker Compose, complete with custom modules and advanced configurations.

For a more basic setup using Docker without custom NPM modules, see How to install n8n on a Local Server and Access it Securely from Anywhere

Prerequisites

• Docker and Docker Compose installed on your system
• Basic understanding of container orchestration
• Server with sufficient resources (minimum 2GB RAM recommended)
• Domain name configured with DNS records (for HTTPS access)

Why You Might Need to Use Docker Compose

Docker Compose becomes essential when you need to extend n8n’s capabilities with custom NPM modules. For example, in our implementation, we needed several specific tools:

• Content conversion tools (@tryfabric/martian and notion-to-md) for transforming Notion pages to JSON and HTML for websites
• Notion API client (@notionhq/client) for direct integration with Notion
• Markdown processing (markdown-it) for content formatting
• Media processing tools (ffmpeg) for extracting audio from video calls for transcription

By using Docker Compose, you can easily manage these dependencies in a reproducible way, ensuring that all necessary tools are properly installed and configured in your n8n environment. This approach offers flexibility to add any NPM packages you need for your specific automation workflows.

Project Structure Setup

First, let’s create a proper directory structure for our n8n deployment:

mkdir n8n-docker
cd n8n-docker
touch docker-compose.yml
touch Dockerfile
touch .env

Creating the Dockerfile

The Dockerfile allows us to extend the base n8n image with custom NPM packages:

FROM n8nio/n8n:latest

USER root

# Install additional npm packages
RUN npm install -g npm \
    @tryfabric/martian \
    notion-to-md \
    # Add other packages as needed

USER node

Docker Compose Configuration

Create a comprehensive docker-compose.yml file that includes all necessary configurations:

version: '3.8'

services:
  n8n:
    container_name: n8n
    build: .
    restart: always
    ports:
      - "5678:5678"
    environment:
      - NODE_FUNCTION_ALLOW_EXTERNAL=*
      - N8N_HOST=${N8N_HOST}
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - WEBHOOK_URL=https://${N8N_WEBHOOK_URL}/
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      # Email configuration
      - N8N_EMAIL_MODE=${N8N_EMAIL_MODE}
      - N8N_SMTP_HOST=${N8N_SMTP_HOST}
      - N8N_SMTP_PORT=${N8N_SMTP_PORT}
      - N8N_SMTP_USER=${N8N_SMTP_USER}
      - N8N_SMTP_PASS=${N8N_SMTP_PASS}
      - N8N_SMTP_SENDER=${N8N_SMTP_SENDER}
    volumes:
      - n8n_data:/home/node/.n8n
      - ${DATA_FOLDER}/files:/files
    networks:
      - n8n-network

networks:
  n8n-network:
    driver: bridge

volumes:
  n8n_data:
    external: true

Important: The environment variable NODE_FUNCTION_ALLOW_EXTERNAL=* is crucial for this setup to work properly. This setting allows n8n to use external NPM modules in your workflows. Without this configuration, any custom NPM packages installed in the Dockerfile will not be accessible within n8n functions.

Make sure to include NODE_FUNCTION_ALLOW_EXTERNAL=* in your environment variables. This is a required setting when working with custom NPM modules, and your workflows may fail if this is not properly configured.

Environment Configuration

Create a .env file with your specific configurations:

# Base Configuration
DATA_FOLDER=/path/to/your/data
N8N_HOST=your-domain.com
N8N_WEBHOOK_URL=your-domain.com
N8N_ENCRYPTION_KEY=your-secure-encryption-key

# Email Configuration
N8N_EMAIL_MODE=smtp
N8N_SMTP_HOST=smtp.provider.com
N8N_SMTP_PORT=587
N8N_SMTP_USER=your-smtp-user
N8N_SMTP_PASS=your-smtp-password
N8N_SMTP_SENDER=Your Name <name@domain.com>

Deployment Steps

1. Initialize the Volume

Create the external volume for persistent data storage:

docker volume create n8n_data

2. Build and Deploy

Start the n8n container with Docker Compose:

docker compose up -d --build

Maintenance and Updates

Updating n8n

To update your n8n installation:

# Pull the latest images
docker compose pull

# Rebuild and restart containers
docker compose down
docker compose up -d --build

Backup and Restore

Regular backups are crucial. Here’s how to backup your n8n data:

# Backup
docker run --rm -v n8n_data:/source -v $(pwd):/backup alpine tar czf /backup/n8n-backup.tar.gz -C /source .

# Restore
docker run --rm -v n8n_data:/target -v $(pwd):/backup alpine sh -c "cd /target && tar xzf /backup/n8n-backup.tar.gz"

Advanced Configurations

Custom NPM Modules

To add new NPM modules, update your Dockerfile and rebuild:

# Update Dockerfile with new packages
docker compose down
docker compose up -d --build

Troubleshooting

• Container won’t start: Check logs using docker compose logs n8n
• Permission issues: Ensure proper volume permissions and ownership
• Network connectivity: Verify network configuration and firewall settings

Security Best Practices

• Always use HTTPS in production
• Regularly update all components
• Use strong encryption keys
• Implement proper access controls

Conclusion

Running n8n with Docker Compose provides a robust, scalable, and maintainable automation platform. This setup enables you to leverage custom NPM modules while maintaining a production-ready environment with proper security measures and backup procedures.

n8n is an advanced, open-source workflow automation tool designed to facilitate rapid prototyping and streamlined automation of various development processes. Think of it as an open-source, self hosted alternative to Zapier. It’s intuitive, visual interface empowers developers to seamlessly connect APIs, services, and integrations without extensive coding, dramatically accelerating workflow creation and experimentation.

Moreover, n8n supports extensive AI integrations, allowing seamless incorporation of AI functionalities into workflows, unlocking opportunities for sophisticated automation and process optimisation.

Access it Securely from Anywhere Using Docker and Cloudflared

Why We Chose Docker and Cloudflared

We opted to self-host n8n using Docker for the following reasons:

• Complete control over data security and resource management.
• Scalability and ease of deployment.
• Custom integration with additional NPM modules and services such as ffmpeg.

To securely expose our n8n instance externally, we integrated Cloudflared (Cloudflare Tunnel), which:

• Ensures secure, encrypted connections without opening firewall ports directly.
• Simplifies external access via Cloudflare’s robust global infrastructure.
• Enhances security by masking the server’s IP address.

Step-by-Step Installation and Configuration

Step 1: SSH into Your Server

Wondering how to get a server? Read How to deploy an ubuntu server before continuing

ssh ubuntu@your_server_ip

Step 2: Deploy n8n with Docker

Run n8n in a Docker container using a custom environment file:

docker run -d --restart=always --env-file ./.env-n8n --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n

Create and populate your .env-n8n file as follows:

DOMAIN_NAME=n8n.yourdomain.com
GENERIC_TIMEZONE=Europe/Berlin

Step 3: Verify and Manage Your Docker Container

Check logs:

docker logs n8n

Stop your container:

docker stop n8n

Step 4: Update Your n8n Container

Regular updates ensure security and functionality improvements:

docker stop n8n
docker rm n8n
docker run --pull always -d --restart=always --env-file ./.env-n8n --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n

Setting Up Cloudflare Tunnel with Cloudflared

Cloudflare Tunnel (Cloudflared) securely routes traffic through Cloudflare without exposing your server IP directly.

Running Cloudflared in Docker (Recommended Method)

Execute the following command to run Cloudflared in a Docker container with host network mode:

docker run -d --restart=always --network="host" --name cloudflared cloudflare/cloudflared:latest tunnel --no-autoupdate run --token YOUR_CLOUDFLARE_TUNNEL_TOKEN

Managing Cloudflared Container

Stop and remove the Cloudflared container as needed:

docker stop cloudflared
docker rm cloudflared

Why Use Host Network Mode?

• Simplifies the setup by allowing Cloudflared to easily connect with local services.
• Eliminates the complexity of Docker’s internal networking.
• Ideal for setups managing multiple services through Cloudflare Tunnel.

Security Considerations

• Using the host network mode removes Docker’s network isolation, meaning careful consideration of the security context is required.
• Ensure your server is otherwise secure and well-monitored.

How to Access n8n

Your n8n application will be securely accessible at:

https://n8n.yourdomain.com

Application and Utility of n8n

We utilise n8n extensively for:

• Creating agile production workflows with minimal code.
• Rapid prototyping of complex automations.
• Integrating AI into existing and new processes, significantly enhancing our operational efficiency.

Getting Started with n8n

Explore these resources to jumpstart your n8n automation journey:

• Try n8n – Official Guide

Advanced Implementations

We’ve successfully transitioned various n8n workflows into full-fledged applications, including Python-based AI chatbots. Further details on such integrations:

• More posts on advanced implementations coming soon!

Taking things further with Docker Compose

While this guide provides a solid foundation for running n8n using Docker, it only scratches the surface of what’s possible. For a more advanced implementation using Docker Compose with custom NPM modules and enhanced functionality, check out our detailed guide on How to run n8n with docker compose to use custom NPM modules.

The Docker Compose setup enables you to:

• Integrate custom NPM modules like @tryfabric/martian for enhanced capabilities
• Configure complex email settings using AWS SES
• Manage persistent data storage more effectively
• Scale your automation workflows with better resource management

Using this setup, we’ve greatly enhanced our automation capabilities, boosted development efficiency, and maintained optimal security standards. Explore n8n today, and unlock a world of powerful automation possibilities!

Understanding Architectural Decision Records (ADRs)

An Architectural Decision Record (ADR) is a document that captures a significant architectural decision made during a project, along with its context and consequences. This practice provides a structured approach to recording the rationale behind key decisions, facilitating better communication among stakeholders and serving as a valuable reference for future project phases.

Our Approach to ADRs at The Website Engineer

In our commitment to delivering top-tier web solutions, we integrate ADRs into our development workflow as follows:

1. Context Establishment: We begin by clearly defining the problem or requirement that necessitates an architectural decision. This includes understanding the project’s goals, constraints, and the specific challenges we aim to address.
2. Decision Drivers: We identify the key factors influencing our decision, such as scalability, performance, security, and maintainability. Recognising these drivers ensures that our choices align with both project requirements and client expectations.
3. Option Analysis: Our team evaluates multiple architectural options, considering the pros and cons of each. This thorough analysis allows us to make informed decisions that best suit the project’s unique needs.
4. Decision Documentation: Once an option is selected, we document the decision in detail, including the reasoning behind it and any anticipated consequences. This record serves as a transparent reference for all stakeholders.
5. Review and Confirmation: We periodically review our ADRs to ensure they remain relevant and effective as the project evolves. This iterative process allows us to adapt to new insights or changing requirements.

Benefits of Using ADRs

Implementing ADRs within our projects offers several advantages:

• Enhanced Communication: ADRs provide a clear record of decisions, facilitating better understanding among team members and stakeholders.
• Knowledge Sharing: They serve as educational tools for new team members, offering insights into the project’s architectural evolution.
• Informed Future Decisions: Documenting past decisions helps prevent the repetition of discussions and allows for more informed decision-making in future projects.

ADRs in Action: A Practical Example

Consider a scenario where we need to choose a database solution for a client’s expanding user base. An ADR would document the context (e.g., the need for scalability), decision drivers (e.g., data consistency, integration ease), considered options (e.g., PostgreSQL, MongoDB, Amazon DynamoDB), the chosen solution, and the rationale behind it. This structured documentation ensures that all stakeholders understand the decision and its implications.

Here’s our template to create an ADR:

# Example Title: Database Choice for User Data

## Context and Problem Statement

We need a scalable database to store and manage user data efficiently as our user base grows.

## Decision Drivers

* Scalability
* Data consistency
* Ease of integration with existing services

## Considered Options

* PostgreSQL
* MongoDB
* Amazon DynamoDB

## Decision Outcome

Chosen option: **PostgreSQL** because it provides strong data consistency and aligns well with our need for complex queries.

### Consequences

* **Good:** Supports ACID compliance, enhancing data reliability.
* **Bad:** May require more tuning to achieve high performance with large datasets.

### Confirmation

We’ll confirm this decision through periodic load tests and performance reviews as the user base scales.

## Pros and Cons of the Options

### PostgreSQL

* **Good:** ACID compliance, robust community support.
* **Neutral:** Setup and tuning can be time-consuming.
* **Bad:** Lacks native horizontal scaling.

### MongoDB

* **Good:** Schema flexibility, horizontal scaling.
* **Bad:** No ACID compliance across collections, limiting data integrity.

## More Information

For additional details, see the database performance evaluation [here](link-to-evaluation).

Inspiration

Drawing inspiration from AWS’s robust architectural decision-making process, we’ve adapted their systematic approach to ADRs while maintaining our unique perspective at The Website Engineer. Like AWS, we recognize that methodical documentation and clear decision-making processes are fundamental to building scalable, reliable systems. Our ADR implementation reflects AWS’s commitment to well-documented architecture decisions, fostering a culture of intentional design and collaborative problem-solving that has proven successful in large-scale enterprise environments.

https://docs.aws.amazon.com/prescriptive-guidance/latest/architectural-decision-records/adr-process.html

Conclusion

At The Website Engineer, our utilisation of ADRs reflects our dedication to meticulous planning and transparent communication in web development. This practice not only enhances our internal processes but also assures our clients of our commitment to delivering scalable and well-considered solutions. For potential new hires, understanding our ADR process offers insight into our collaborative and methodical approach to tackling architectural challenges.

By embedding ADRs into our workflow, we uphold a culture of continuous improvement and shared knowledge, positioning ourselves to meet the evolving needs of our clients effectively.

If you’re experiencing sluggish performance when developing websites locally, you’re not alone. Slow localhost load times can drastically slow down your development process, creating frustration and reducing productivity. In this comprehensive guide, we’ll explore common reasons why your localhost might be loading slowly, and provide practical solutions to streamline your local development workflow, especially for WordPress.

Common Reasons Why Localhost Loads Slowly

1. Large Database and Heavy Queries

One of the most common reasons for a slow localhost environment is a large database. WordPress sites, in particular, often contain extensive databases that can take significant time to query.

Solution: Optimise your database by regularly clearing unnecessary data, such as spam comments, post revisions, and transient data. Tools like WP-Optimize can automate this process.

2. Unoptimised Images

When developing locally, using uncompressed, high-resolution images can significantly slow down loading times.

Solution: Compress and optimise images before using them locally. Tools such as TinyPNG or Imagify can help drastically reduce file sizes without sacrificing quality.

3. Insufficient Hardware Resources

Limited RAM or CPU resources on your computer can slow your localhost server performance.

Solution: Upgrade your hardware or allocate more resources to your local development environment. Consider using lightweight local development stacks like Local by Flywheel, XAMPP, or Docker, which allow efficient resource management.

4. Misconfigured Local Servers

Incorrectly configured local servers can lead to slow loading times, especially when Apache or PHP settings are not optimised.

Solution: Adjust Apache and PHP configurations. For Apache, tweak settings like KeepAlive, MaxKeepAliveRequests, and Timeout. For PHP, increase the memory limit (memory_limit) and reduce execution time (max_execution_time) appropriately in your php.ini file.

5. External Resource Dependencies

Relying on external resources such as Google Fonts, APIs, or external images from the internet can significantly slow down your localhost environment, especially if you have poor internet connectivity.

Solution: Minimise external dependencies by caching resources locally or using fallback mechanisms. For WordPress images, implementing an Apache fallback solution can resolve image loading issues efficiently.

Streamlining WordPress Local Development with Image Fallbacks

A common pain point for WordPress developers working locally is managing media files. Typically, developers prefer not to download all media assets from the production environment due to time and storage constraints. However, this often leads to broken images on the local site.

Efficient Solution: Apache Image Fallback

To avoid broken images without downloading all media files, implement a simple Apache image fallback. Here’s how:

Add the following snippet to your .htaccess file:

# BEGIN WordPress Image Fallback
<IfModule mod_rewrite.c>
  RewriteEngine On

  # Check if the requested file exists locally
  RewriteCond %{REQUEST_FILENAME} !-f
  # Check if the request is for an image file
  RewriteCond %{REQUEST_URI} \.(jpg|jpeg|png|gif|webp|svg)$ [NC]

  # Redirect request to the live site if image is not found locally
  RewriteRule ^(.*)$ https://yourwebsite.com/$1 [L,R=302]
</IfModule>
# END WordPress Image Fallback

Replace yourwebsite.com with your live site’s domain.

Benefits:

• Speeds up local site loading times significantly
• Reduces disk space usage
• Preserves full site functionality during development
• Provides live site images without the hassle of manual syncing

FAQs on Localhost Loading Slowly

Q1: Why is my localhost slower than my live website?

Your localhost might be slower due to limited local hardware resources, unoptimised settings, or external dependencies loading slowly over your internet connection.

Q2: Can antivirus software slow down localhost performance?

Yes, antivirus software can interfere by scanning files accessed by your local server. Configure your antivirus to exclude scanning your localhost directory to improve performance.

Q3: Does browser cache affect localhost speed?

Yes, browser caching settings affect localhost speed. Clear browser cache regularly or disable caching during active development to ensure you’re seeing real-time changes.

Q4: How much RAM should I allocate for local WordPress development?

For smooth performance, allocate at least 4 GB RAM. Complex sites might require 8 GB or more for optimal performance.

Q5: Is it necessary to optimise images for local development?

Yes, optimising images reduces loading times significantly, allowing quicker testing and improving your development workflow.

Conclusion

Improving localhost performance involves addressing various factors, including database optimisation, server configuration, hardware allocation, and managing external dependencies. By implementing the solutions discussed in this guide, you can create a more efficient local development environment, boosting productivity and streamlining your workflow significantly.