# Configure S3 or Minio for Storing Execution Logs

Rundeck execution logs are a set of files generated after each job execution. By default, Rundeck stores them on the local filesystem. Another way to store these logs is on a dedicated file storage service like AWS S3 (opens new window).

By delegating this storage to AWS S3 (or any other S3-compatible bucket system like Minio (opens new window)) it’s possible to use some features (opens new window) like S3 replication, security management, data persistence, and saving space on the local filesystem.

In PagerDuty Process Automation (formerly “Rundeck Enterprise”) cluster, S3-compatible storage should be the default configuration to ensure access to logs from any cluster member.

This article explains how to configure Rundeck so that these execution logs are stored on services such as Amazon S3 or Minio.

# Steps to configure S3 execution logs on Rundeck OSS / Process Automation for Global Config

  1. Stop the Rundeck service
    systemctl stop rundeckd
    
  2. S3 Log Storage Plugin
  3. To enable the S3 / Minio log storage plugin, add the following line on the rundeck-config.properties file
  4. Then open the framework.properties file and add the S3 / Minio bucket info as follow:

Definitions:

  • endpoint: A custom S3 compatible endpoint to use, such as https://my-host.com/s3
  • pathStyle: Optional, boolean, default=False, set to True if you need to define the bucket in your S3 like endpoint URL. e.g: https://<s3_like_end_point_url\>/<your_bucket_name\>
  • AWSAccessKeyId: AWS access key, required if using AWSSecretKey.
  • AWSSecretKey: AWS secret key, required if using AWSAccessKeyId.
  • AWSCredentialsFile: Properties file which contains accessKey and secretKey entries. The alternative to specifying the AWSAccessKeyId and AWSSecretKey
  • bucket: The name of the S3 bucket to use. This is the shorthand name, eg test-rundeck-logs
  • region: The Region your Instance is located, eg us-east-1

# Test basic setup

  1. Start the Rundeck service.
  2. Create a new project.
  3. Create a new job and then execute it, now the execution log is stored in the S3/Minio bucket.

  4. From Rundeck click on the Gear Icon and then to the Log Storage option, now you can see the Log Storage Activity.

# PagerDuty Process Automation (formerly “Rundeck Enterprise”) System Configuration

  1. As admin rights users click on the Gear Icon and then click on “System Configuration”.

  2. Then click on the “+ Add Config” button.

  3. Add the following property rundeck.execution.logs.fileStoragePlugin with the following value com.rundeck.rundeckpro.amazon-s3.

  4. Add the following custom properties:

  5. Restart the PagerDuty Process Automation service.

# Steps to configure S3 execution logs on Rundeck OSS / Process Automation for individual projects

  1. As admin rights users click on Project Settings then Edit Configuration and then Edit Configuration File

  2. Then add these lines with your information:
project.plugin.ExecutionFileStorage.com.rundeck.rundeckpro.amazon-s3.AWSAccessKeyId=your_aws_access_key
project.plugin.ExecutionFileStorage.com.rundeck.rundeckpro.amazon-s3.AWSSecretKey=your_aws_secret_key
project.plugin.ExecutionFileStorage.com.rundeck.rundeckpro.amazon-s3.bucket=your_s3_bucket_name
project.plugin.ExecutionFileStorage.com.rundeck.rundeckpro.amazon-s3.path=logs/${job.project}/${job.execid}.log
project.plugin.ExecutionFileStorage.com.rundeck.rundeckpro.amazon-s3.region=your_instance_region
  1. Then click Save button.
  2. Restart the PagerDuty Process Automation service.

# Docker Config

# Resources

Last Updated: 9/28/2023, 11:38:04 PM