Managing MongoDB with Runbook Automation
Managing MongoDB with Runbook Automation
MongoDB is a popular NoSQL database that stores data in flexible, JSON-like documents. This guide demonstrates how to use Rundeck's MongoDB Command Step Plugin to automate database operations, manage collections, and integrate MongoDB operations into your automation workflows.
This article assumes you're using Runbook Automation SAAS. The MongoDB plugins will be fully implmented in version 5.13.0 for the Self Hosted product.
Prerequisites for This Tutorial
- MongoDB Server or MongoDB Atlas account
- Either a MongoDB server installation
- Or a MongoDB Atlas account with a cluster
- MongoDB database credentials
- Username and password with appropriate permissions
- Network access
- Ensure Rundeck server can reach your MongoDB instance
- PagerDuty Runbook Automation v5.11.1+
Local MongoDB Setup Example
First, install MongoDB on your server. For Ubuntu systems:
sudo apt-get install mongodb
Next, create an admin user by connecting to MongoDB and running:
use admin
db.createUser({
user: "admin",
pwd: "securePassword",
roles: [ "userAdminAnyDatabase" ]
})
Then create a sample database and collections:
use ecommerce_db
db.products.insertMany([
{
name: "Laptop",
price: 999.99,
category: "Electronics",
stock: 50
},
{
name: "Smartphone",
price: 699.99,
category: "Electronics",
stock: 100
}
])
Store Password in Key Storage
- Open Key Storage and save the "securePassword" value from the above set in as a Password value entry named
mongodb
.
Configuring the MongoDB Command Step
- Navigate to your job definition and add a new step
- Select "MongoDB Command" from the step picker
- Configure the connection settings:
For a basic MongoDB step configuration:
- Database Deployment: "MongoDB"
- Server: "yourMongoServer"
- Port: "27017" (or leave blank as this is the default)
- Database: "ecommerce_db"
- Username: "admin"
- Password: keys/mongodb
- Command:
{
find: "products",
filter: { "category": "Electronics" }
}
Common Use Cases and Examples
1. Basic Query Operations
To query all products in a collection:
{
find: "products",
filter: {},
limit: 10
}
2. Aggregation Pipeline
To calculate total inventory value:
{
aggregate: "products",
pipeline: [
{
$group: {
_id: null,
totalValue: {
$sum: { $multiply: ["$price", "$stock"] }
}
}
}
],
cursor: {}
}
3. Update Operations
To update product stock levels:
{
update: "products",
updates: [
{
q: { name: "Laptop" },
u: { $set: { stock: 45 } }
}
]
}
Please follow the Basic Example first
Provided below is a multi-step job definition for setting up a complete e-commerce database example. This job will include steps for creating collections, inserting data, and creating indexes.
Assumptions made in the job steps:
- MongoDB host name is configured as
mongodb
- MongoDB user name is configured as
admin
- Password is stored in Key Storage at
keys/mongodb
This job definition includes:
Seven sequential steps that:
- Create and populate the products collection
- Create and populate the customers collection
- Create and populate the orders collection
- Create indexes for all collections
- Verify the setup with a count check
To use this job:
- Save it as a YAML file (e.g., mongodb-ecommerce-setup.yaml)
- Import it into your Rundeck instance
- Ensure the MongoDB password is stored in the specified key storage path
- Ensure that
mongodb
hostname is resolvable, if not update the steps with the right hostname. - Update the mongoHost value to match your MongoDB server
- Run the job
The job will execute each step in sequence, creating a complete e-commerce database setup. The final step verifies the setup by counting the products collection.
- defaultTab: nodes
description: 'Sets up a complete e-commerce database with products, customers, and orders'
executionEnabled: true
group: MongoDB
id: mongodb-ecommerce-setup
loglevel: INFO
name: MongoDB E-Commerce Setup
nodeFilterEditable: false
nodefilters:
dispatch:
excludePrecedence: true
keepgoing: false
rankOrder: ascending
successOnEmptyNodeFilter: false
threadcount: '1'
filter: 'tags: "RUNNER"'
nodesSelectedByDefault: false
scheduleEnabled: true
sequence:
commands:
# Step 1: Create Database
- configuration:
mongoCommand: |-
{
"dropDatabase": 1
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Drop existing database if exists
nodeStep: true
type: mongodb-nodestep
# Step 2: Create Products Collection
- configuration:
mongoCommand: |-
{
"create": "products"
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Create Products Collection
nodeStep: true
type: mongodb-nodestep
# Step 3: Insert Products
- configuration:
mongoCommand: |-
{
"insert": "products",
"documents": [
{
"name": "Laptop",
"price": 999.99,
"category": "Electronics",
"stock": 50,
"specifications": {
"brand": "TechBrand",
"model": "Pro2024",
"ram": "16GB",
"storage": "512GB SSD"
}
},
{
"name": "Smartphone",
"price": 699.99,
"category": "Electronics",
"stock": 100,
"specifications": {
"brand": "MobileX",
"model": "X12",
"ram": "8GB",
"storage": "256GB"
}
},
{
"name": "Coffee Maker",
"price": 79.99,
"category": "Appliances",
"stock": 30,
"specifications": {
"brand": "HomeBrew",
"model": "CM100",
"capacity": "12 cups"
}
}
]
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Insert Products
nodeStep: true
type: mongodb-nodestep
# Step 4: Create Customers Collection
- configuration:
mongoCommand: |-
{
"create": "customers"
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Create Customers Collection
nodeStep: true
type: mongodb-nodestep
# Step 5: Insert Customers
- configuration:
mongoCommand: |-
{
"insert": "customers",
"documents": [
{
"firstName": "John",
"lastName": "Doe",
"email": "john.doe@email.com",
"address": {
"street": "123 Main St",
"city": "Boston",
"state": "MA",
"zipCode": "02108"
},
"phoneNumber": "555-0123"
},
{
"firstName": "Jane",
"lastName": "Smith",
"email": "jane.smith@email.com",
"address": {
"street": "456 Oak Ave",
"city": "Chicago",
"state": "IL",
"zipCode": "60601"
},
"phoneNumber": "555-0124"
}
]
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Insert Customers
nodeStep: true
type: mongodb-nodestep
# Step 6: Create Orders Collection
- configuration:
mongoCommand: |-
{
"create": "orders"
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Create Orders Collection
nodeStep: true
type: mongodb-nodestep
# Step 7: Create Indexes
- configuration:
mongoCommand: |-
{
"createIndexes": "products",
"indexes": [
{
"key": { "name": 1 },
"name": "idx_product_name"
},
{
"key": { "category": 1 },
"name": "idx_product_category"
}
]
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Create Product Indexes
nodeStep: true
type: mongodb-nodestep
- configuration:
mongoCommand: |-
{
"createIndexes": "customers",
"indexes": [
{
"key": { "email": 1 },
"name": "idx_customer_email",
"unique": true
}
]
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Create Customer Indexes
nodeStep: true
type: mongodb-nodestep
- configuration:
mongoCommand: |-
{
"createIndexes": "orders",
"indexes": [
{
"key": { "orderId": 1 },
"name": "idx_order_id",
"unique": true
},
{
"key": { "customerId": 1 },
"name": "idx_customer_id"
}
]
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Create Order Indexes
nodeStep: true
type: mongodb-nodestep
# Step 8: Verify Setup
- configuration:
mongoCommand: |-
{
"find": "products",
"limit": 1
}
mongoDatabase: ecommerce_db
mongoHost: mongodb
mongoType: MongoDB
mongoUser: admin
outputMode: RELAXED
passwordStoragePath: keys/mongodb
description: Verify Products
nodeStep: true
type: mongodb-nodestep
keepgoing: false
strategy: node-first
tags: 'ecommerce,mongodb,setup'
uuid: mongodb-ecommerce-setup
Best Practices
Security
- Store MongoDB passwords in Rundeck's Key Storage
- Use specific database users with minimum required permissions
- Enable SSL/TLS for database connections
Error Handling
- Add error handlers to your jobs
- Use job references for common error scenarios
- Log important database operations
Performance
- Use appropriate indexes for your queries
- Limit result sets when possible
- Use projection to retrieve only needed fields
Common issues and solutions:
Connection Failed
- Verify network connectivity
- Check credentials
- Confirm port accessibility
Command Execution Errors
- Validate JSON syntax
- Check collection names
- Verify user permissions
Performance Issues
- Review query structure
- Check for missing indexes
- Monitor connection pool settings
Tips
When working with MongoDB Atlas, ensure your Rundeck server's IP address is whitelisted in the Atlas network access settings.