Rundeck can integrate with external data by configuring the use of Providers or Sources. Providers are third-party services or systems that export data that Rundeck can import. Additionally, Rundeck supports an external Editor for Node data.
Rundeck makes use of common data formats (XML, JSON & YAML). Third-party software may produce these formats natively, however it is typical to have to massage the output of one system into the appropriate format to be consumed by Rundeck. Since URLs and HTTP are a lowest-common-denominator for communication, Rundeck only requires that the data Providers make this data available as a file at a URL or on the local disk.
There are a few types of external integration:
Provides a set of Nodes in XML or YAML format. E.g. a CMDB or hosted virtual machines service. Rundeck can be configured to use a different provider for each Project, and can refresh the Resources it uses from this provider.
Provides a web-based editor to manage the Node definitions. Rundeck can link to this editor from the Run page, and has optional JavaScript interactions to make editing externally-managed Node resources integrated with the Rundeck GUI.
The Resource model source is a way to transfer Node definitions from other systems, tools or services into Rundeck. The means of providing the Resource model da ta can be done in whichever way suits your environment best.
Rundeck supports plugins to provide different kinds of sources, but has built-in support for URLs or File based sources.
Resource model data is a set of Node descriptors, each with a uniquely identifying name. In addition to Name, some pieces of metadata are required (like hostname, and username), and some are optional.
(See [Resource Model Document Formats]) for more information on what format the files need to be in.)
The Resource model data is stored on the server as a set of files. Each Project in Rundeck has at least a single Resources file, and may have multiple additional sources (such as a URL or a directory containing multiple files). All of these sources will be combined into the set of all Nodes that are available for the project. Each node's metadata can define how to connect to it and run commands.
In order to provide the Resource model data to Rundeck:
name value. You may have to convert the external system's identifier to be unique, or create one yourself.This means you can provide the data in the way that best suits your specific use-case. Some examples:
The Resource model data does not have to include the Rundeck server Node, as it is implicitly included.
Resource model sources are defined on a per-project basis, in the project.properties file.
The only required configuration value is project.resources.file, which defines a single file containing resource model data stored on-disk. Each new project will have a good default location, but you may change either the location or the file extension. The file extension determines the format of the data. (See [Resource Model Document Formats].)
project.resources.file = ..This file path is where Rundeck will read the contents from, and also where it will store it to if refreshing from a remote URL.
You may also specify a URL, which will be automatically retrieved and stored in a cache file. The Resource Model data within the content is merged with the previous file.
project.resources.url = http://...This configures the remote URL for loading the Resources model data.
In addition, multiple [Resource Model Source Plugin] can be configured to add additional sources of Resource Model data.
The ordering of sources defines the order that node definitions are retrieved. By default in Rundeck 2.0+, any identically named nodes that are reported by multiple sources have all of their attributes merged, with later attribute values overriding earlier values.
To disable attribute merging, you can set project.resources.mergeNodeAttributes=false in project.properties. This means that later node definitions completely override any earlier definitions with the same node name.
Putting the resources.xml/resources.yaml file under the control of a source code management tool is a simple solution to controlling and tracking change to the resources.xml file. Any changes will be committed and the commit messages become an audit log. Most source code management tools provide a web interface to retrieve file revisions based on a URL and thus make it accessible as a resource model source.
Going back to the Tutorial chapter, imagine the administrator decides the VCS approach is a good first step to control versioning for the anvils resource model. Acme is a subversion user and installed viewvc to give web access to the repository.
First, the current resources.xml is added to the repository and committed:
svn add resources.xml http://svn.acme.com/ops/anvils/resources.xml
svn commit -m "added resource model for anvils" resources.xmlTo test access, the administrator downloads the latest revision (ie, "HEAD") via the "viewvc" interface.
curl http://svn.acme.com/viewvc/ops/anvils/resources.xml?revision=HEADNext, the anvils project.properties configuration file is modified to reference the URL to retrieve the "HEAD" revision:
project.resources.file = /etc/rundeck/projects/anvils/resources.xml
project.resources.url = http://svn.acme.com/viewvc/ops/anvils/resources.xml?revision=HEADThis configuration specifies the anvils resource model will be retrieved from project.resources.url and then stored at project.resources.file. Now, anytime Rundeck refreshes the anvils resource model, it will request the resources.xml file from the viewvc URL, obtaining the latest revision.
Amazon's EC2 (Elastic Cloud Compute) is a cloud service in wide use for dynamic infrastructure; it is easy to start up and shut down Node "Instances" in the cloud.
For Rundeck, we would like to have a way of querying the EC2 service to see what EC2 Instances are available for use as Rundeck Nodes.
Amazon has a well-defined API for communication with their services, which would allow us to pull out the EC2 data, and generate XML if we wanted to. We could write a script that produces that data and use that script on a server to produce data via a URL, or we could use that script with the script resource model source plugin to generate it. This would give us complete control of the output, but does require extra work.
However, there is already a plugin to do this for you: the Rundeck EC2 Nodes Plugin.
Use is fairly simple:
$RDECK_BASE/libext directory. The plugin contains all of the required dependencies.Minimal configuration details for the plugin includes your AWS access credentials you can find here http://aws.amazon.com/security-credentials.
Specify your AWS Access key.
Specify your AWS Secret Key
Read about the other configuration details in the readme for the rundeck-ec2-nodes-plugin.
Finally, within Rundeck, you can Refresh the Nodes from within the Run tab. You should see a Node entry for each EC2 Instance that is available.
You can manage the set of Nodes that gets returned from the plugin by organizing your EC2 instances using EC2 Tags, as well as adding EC2 Filters to the plugin configuration.
The EC2 plugin will automatically add tags for the nodes based on an EC2 Instance Tag named "Rundeck-Tags", as well as the Instance's state. You can also add "Mapping parameters" to the EC2 Plugin configuration to add additional tags.
You can add filters to the EC2 Plugin configuration under the "Filter Params" configuration area, with the sytanx of: filter=value;filter2=value2. The available filter names are listed in AWS API - DescribeInstances.
You can also configure your EC2 Plugin manually or automatically by creating or modifying the project.properties file, and defining a Resource Model Source provider, like this:
resources.source.1.type=aws-ec2
resources.source.1.config.accessKey=...
resources.source.1.config.privateKey=...
resources.source.1.config.filter=...More configuration is available for the rundeck-ec2-nodes-plugin project.
URL Resource model sources can be developed by third parties to integrate Rundeck with their tools.
Check the list on our wiki: https://github.com/rundeck/rundeck/wiki/Resource-model-providers.
The Resource Editor integration is a way to link to a third-party system used for managing Node definitions from within Rundeck. Each Node entry in the resources.xml or resources.yaml can define a URL to provide an "Edit" link that will appear in the Rundeck Run page for that Node.
This allows you to make use of the Resource Model Source in a more seamless way. Rundeck will load the Resource model from the third-party Provider system, and users of Rundeck can click straight to the Editor for those Nodes. The Provider and the Editor could be the same system, or they could both be custom CGI scripts that integrate with a third system.
Some teams have acquired or developed tools to manage information about the hosts deployed in their networks. These tools have interfaces to not just view but also modify the data about these hosts. Though there is no widely used common standard adopted by users of these tools, it is possible to map the data to meet the needs of Rundeck resource models.
The Rundeck resource model document format and the resource-yaml-v13 format provide two attributes that help connect the dots between the Rundeck UI and the editing interface provided by the external data management tool. They can use editUrl or remoteUrl attributes to specify the remote URL. The URLs can embed properties about the node to expand prior to being loaded, which allows you to e.g. submit query parameters using the node name.
editUrlSpecifies a URL to a remote site which will allow editing of the Node. When specified, the Node resource will display an "Edit" link in the Rundeck GUI and clicking it will open a new browser page for the URL.
remoteUrlSpecifies a URL for a remote site which will be loaded in an iframe within a Rundeck page. Clicking the "Edit" link for the Node will load content from the site within the current Rundeck page, allow you to perform your edit at the remote site, and has optional JavaScript hooks to report the state of the editing process back to the Rundeck page for a more streamlined user interface.
Properties of the Node can be embedded in the URL and expanded prior to use. The syntax is:
${node.property}Available properties are:
name, hostname, os-name, os-version, os-family, os-arch, username, description, tags, project
You can embed these properties within the url like this:
http://mycmdb:8080/node/edit?name=${node.name}Using the remoteUrl lets you embed another site into an iframe within the Rundeck page, and optionally allows communication back to the Rundeck page about the state of the editing process.
If you want to embed the remote site without having to make any changes to the remote page content, you can do so simply by specifying the remoteUrl to use. When the user clicks "Edit" the site will load within an iframe, and the user can perform whatever actions on the site are necessary. After they are done they will have to manually click the "Close" button on the Rundeck page to close the iframe.
If you want the user interface in Rundeck to be more streamlined, you will have to be able to modify the web pages produced by the remote site to add simple Javascript calls to communicate back to the Rundeck page. The JavaScript hooks are designed to not add much burden to the developer of the remote site or system, so they are optional.
If the remote site implements some Javascript messaging conforming to a simple optional protocol, then the user interface between Rundeck and the remote site can be made more seamless.
Rundeck lets the remote site inform it when the following steps occur:
Due to web browser security restrictions, direct communication between different webpages can only be done through use of the postMessage method.
The remote page can send these messages simply with this javascript:
<script type="text/javascript">
if(window.self!=window.parent){
window.parent.postMessage("...message...","http://rundeckserver:port");
}
</script>window.parent will be the enclosing browser window when the site is loaded within an iframe. This script simply checks whether the page is loaded in an iframe before sending the message.
The first argument to postMessage is one of the message codes shown below. The second argument is the expected "origin", meaning the URL scheme, server and port of the server receiving the message. You can specify "*" to include any site that may be loading the content, but you may want to restrict it to your Rundeck installation's scheme, hostname and port.
Rundeck can receive the following messages sent by the remote site:
rundeck:node:edit:startedSent as soon as the remote edit URL is loaded and indicates that the remote Site understands the messaging protocol and has loaded the correct edit page. You would probably send this on the "edit" or "form" page for the targetted node.
rundeck:node:edit:error or rundeck:node:edit:error:An error messageSent if some error occurs. The remote editing form will close and the error message (if any) will be shown. You would probably send this on the "edit" or "view" page if there is an error locating the targeted Node or loading anything required for the edit process.
The next two messages are only valid after the "started" message has already been received:
rundeck:node:edit:finished:trueSent after the remote form has been saved without errors. This indicates that the editing process is done and has completed with saved changes. You would probably send this on the "view" or "show" page for the targeted node if the save operation was successful.
rundeck:node:edit:finished:falseSent after the remote form has been either cancelled or discarded without changes. This indicates that the editing process is done but no changes were made. You would probably send this on the "view" or "show" for the targeted node (if your site simply shows the node view again) or "list" page (if your site goes back to a list of resources) if the user hits "cancel".
Any message not shown here that is received by Rundeck after it has received the "started" message will be considered unexpected and the editing process will close the iframe.
The user will also have the option to close and cancel the remote editing process at any time.
Note that sending the "error" or "finished" message will close the editing session and all subsequent messages will be ignored.
TODO: The JavaScript code to communicate back to Rundeck could be bundled into a simple widget script file for easier inclusion on remote sites.
Here are some examples of using the editUrl and remoteUrl in a resources.xml/resources.yaml file:
Specify a simple URL for editing, which will simply produce a link:
<node name="venkman" editUrl="http://mycmdb:8080/node/edit" ... />Specify a URL for editing, with embedded "name" property as a parameter:
<node name="venkman" editUrl="http://mycmdb:8080/node/edit?name=${node.name}" ... />Specify a remote URL with embedded "name" and "project" properties as parameters:
<node name="venkman" remoteUrl="http://mycmdb:8080/node/edit?name=${node.name}&project=${node.project}" ... />Specify a remote URL with embedded "name" property as part of the path:
<node name="venkman" remoteUrl="http://mycmdb:8080/node/edit/${node.name}" ... />In YAML, some examples:
Specify a remote URL with embedded "name" and "project" properties as parameters:
venkman:
nodename: venkman
remoteUrl: http://mycmdb:8080/node/edit?name=${node.name}&project=${node.project}Specify a remote URL with embedded "name" property as part of the path:
venkman:
nodename: venkman
remoteUrl: "http://mycmdb:8080/node/edit/${node.name}The ndbtest project on github provides an example of how the remote Resource Editor can integrate with Rundeck using JavaScript.
This project is a simple Grails application which provides a database of Node data. The standard web-based user flow is:
We want the Node's "edit" link in Rundeck to go directly to an edit page, so the remoteUrl for each Node entry then should be a URL to link to this page, for example:
remoteUrl="http://localhost:8080/node/edit?name=${node.name}&project=${node.project}"The code below shows that the name & project are used to select the correct node from the database, even though the built-in identifier is an ID number:
So the JavaScript for integrating with Rundeck is then added to the following pages in this system:
started message starting on line 34To complete the round-trip of editing a Node and then showing the results back in Rundeck, the ndbtest project would have to export XML formatted Resource data, and then your Rundeck project.properties file would have to point to the appropriate URL. (This is left as an exercise to the reader.)