Plugins
Polemarch can be extended with different type of plugins. This manual describes how to create, setup and start using your first Execution or Inventory plugin and provides full API reference to learn more about them.
Execution plugins
Execution plugins system allows you to execute any external command from Polemarch. To create an execution plugin you need to
create a python class describing the argument processing and API talking logic;
configure this plugin in your
settings.ini
.
Warning
Since v3.0.0 path to execution plugins has been changed from polemarch.plugins
to
polemarch.plugins.execution
.
Execution plugins: quick start
All built-in execution backends, such as ansible playbook or module execution, use plugins (starting from 2.2.0).
To get started let’s create a simple plugin allows you to run echo
command.
from rest_framework.fields import BooleanField, empty
from vstutils.api.fields import VSTCharField
from polemarch.plugins.execution.base import BasePlugin
# Any plugin must be inherited from BasePlugin
class TestEcho(BasePlugin):
# Define fields which will be used to execute this plugin and create template with
serializer_fields = {
# You can use fields either from rest_framework or vstutils
'string': VSTCharField(),
'n': BooleanField(default=empty, required=False, label='No trailing newlines'),
'e': BooleanField(default=empty, required=False, label='Interpret backslash escapes'),
}
# Value of this field will be shown on history detail page as Mode
arg_shown_on_history_as_mode = 'string'
# This is our binary from which any command starts
@property
def base_command(self):
return ['echo']
# This method is called by get_args method of BasePlugin for each argument received from API. As we defined
# 'string', 'n', and 'e' arguments in serializer_fields, we can get their keys and values here.
def _process_arg(self, key, value):
# As 'string' argument in this case is a positional argument, let's return just it's value, so the final
# command will be something like ['echo', 'string to output', ...]
if key == 'string':
return value
# As this guys are just boolean flags, let's return them as '-n' or '-e' accordingly
if key in ('n', 'e') and value:
return f'-{key}'
# Note, that if we received for example `n` argument with False value, we are returning None,
# means that it won't be included to execution command. But of course, you may override this behavior
# in get_args method.
Supposing that described plugin is located at polemarch.plugins.execution.custom.Echo
, let’s connect it to
Polemarch. In your /etc/polemarch/settings.ini
add following section:
[execution.plugin.echo]
backend = polemarch.plugins.execution.custom.Echo
Also you may want to provide additional options directly to plugin:
[execution.plugin.echo.options]
some_option = 'some_option'
In this example some_option will be available in any plugin’s instance method as self.config['some_option']
, as
all options are initialized in the __init__
method.
So it’s all done! After restarting your polemarch server and resetting schema, you can check
#/project/<your_project_id>/execute_echo/
. Here you should be able to execute echo plugin as any built-in one.
Also you should be able to create a template with it at #/project/<your_project_id>/execution_templates/new/
.
If you tried executing echo plugin with flags, you may see that this flags are being outputted too. This is because
they goes after string
argument. To fix this issue, we may do something like this:
...
class TestEcho(BasePlugin):
...
def get_args(self, raw_args):
# We know that 'string' is required so no default for .pop() is needed
string_value = raw_args.pop('string')
args = super().get_args(raw_args)
# Push our string to the end of command
args += [string_value]
return args
@property
def base_command(self):
return ['echo']
def _process_arg(self, key, value):
if key in ('n', 'e') and value:
return f'-{key}'
Now if you are passing flags to execution, they should work the same except not being outputted.
Note
If your execution plugin may work with inventories, you should specify which inventory plugins are compatible with your execution plugin. By default it’s assumed that execution plugin can’t work with inventories. For more information about inventory plugins please see Inventory plugins.
To learn more about what plugins are provide, please check API reference.
Inventory plugins
Inventory plugins system allows you to define how inventory stores, manages and displays its state. To create an inventory plugin you need to
create a python class which manages inventory state and API talking logic;
configure this plugin in your
settings.ini
.
Inventory plugins: quick start
To get started let’s create a simple plugin which works like a simplified version of built-in AnsibleString
with
some additional changes.
from vstutils.api import fields as vstfields
from polemarch.plugins.inventory.base import BasePlugin
class CustomAnsibleString(BasePlugin):
# Our plugin will support import and we will implement corresponding method
supports_import = True
# These fields will be used for working with inventory state in API
serializer_fields = {
# You can use fields either from rest_framework or vstutils
'body': vstfields.TextareaField(allow_blank=True, default=''),
}
# Default values for our fields which will be used to initialize inventory state
defaults = {
'body': 'localhost ansible_connection=local',
}
# These fields will be used for import action in API
serializer_import_fields = {
'body': vstfields.FileInStringField(),
}
def render_inventory(self, execution_dir):
# Getting inventory state. This is possible because by default state_managed attribute of plugin class
# is True
state_data = self.instance.inventory_state.data
filename = str(uuid1())
# Any created files must be in execution_dir
filepath = Path(execution_dir) / filename
filepath.write_text(state_data['body'])
# We doesn't need any additional files so the second argument is empty list
return filepath, []
def get_raw_inventory(self, inventory_string):
# This string will be shown on history page
return f'File contents:\n{inventory_string}'
@classmethod
def import_inventory(cls, instance, data):
# Here we got data which structure corresponds to serializer_import_fields
# and created inventory instance. It's recommended to always use update_inventory_state method
# rather than accessing inventory state directly.
instance.update_inventory_state(data=data)
return instance
Supposing that described plugin is located at polemarch.plugins.inventory.custom.CustomInventoryString
, let’s
connect it to Polemarch. In your /etc/polemarch/settings.ini
add following section:
[inventory.plugin.custom_inventory_string]
backend = polemarch.plugins.inventory.custom.CustomInventoryString
Also you may want to provide additional options directly to plugin:
[inventory.plugin.custom_inventory_string.options]
some_option = 'some_option'
In this example some_option will be available in any plugin’s instance method as self.options['some_option']
, as all options are initialized in the __init__
method.
To start working with the created plugin we also need to allow some execution plugin work with this one. Let’s say that ANSIBLE_MODULE plugin can play with our new CUSTOM_ANSIBLE_STRING:
[execution.plugin.ansible_module.options]
; Make sure you not disabled other built-in inventory plugins
compatible_inventory_plugins = polemarch_db,ansible_file,ansible_string,custom_inventory_string
Done! Now you can create inventory with your plugin at #/project/<your_project_id>/inventory/new/
,
execute ANSIBLE_MODULE plugin at #/project/<your_project_id>/execute_ansible_module/
selecting created inventory.
API reference
- class polemarch.plugins.execution.base.BasePlugin(options=None, output_handler=None)
Base execution plugin class from which any other plugin should inherit. The plugin itself is an entity which, on the one hand provides appropriate fields for the API, and on the other hand, processes arguments received from it to build execution command.
For each configured plugin an endpoint will be generated allows you to choose arguments and execute it. Also, this plugin will be available to create template with.
- Parameters
options (
dict
) –settings.ini
options mapping for this plugin.output_handler (
typing.Optional
[typing.Callable
]) – executor’s function which handles logging and outputting to history. Used byverbose_output
method.
- _get_serializer_fields(exclude_fields=())
Returns field name and field instance mapping used to generate fields for serializer.
- Parameters
exclude_fields (
tuple
) – field names that should not be presented in serializer.- Return type
typing.Mapping
[str
,rest_framework.fields.Field
]
- _get_serializer_metaclass(exclude_fields=())
Returns serializer metaclass used to generate fields in serializer.
- Parameters
exclude_fields (
tuple
) – field names that should not be presented in serializer.- Return type
typing.Type
[typing.Type
[vstutils.api.serializers.BaseSerializer
]]
- _process_arg(key, value)
Returns single argument with value for
get_args
method. Should returnNone
if argument must not be included to the execution command.- Parameters
key (
str
) – argument key (e.g. verbose).value (
typing.Any
) – argument value (e.g. 2).
- Return type
-
arg_shown_on_history_as_inventory:
typing.Optional
[str
] = None Name of argument presented in generated serializer which will be shown on list history page as Inventory.
-
arg_shown_on_history_as_mode:
typing.Optional
[str
] = None Name of argument presented in generated serializer which will be shown on detail history page as Mode. For example, if you are executing some module with additional arguments and fields contains module field, than you can set ‘module’ here, and it’s value will be shown after execution. If not set, Mode in the history will show
[<plugin name> plugin]
string.
-
base_command:
typing.List
[str
] Base command (usually binary) from which execution command starts, e.g.
['echo']
. You may also override this attribute as a property if more complex logic needs to be performed.
-
error_codes:
typing.Mapping
[int
,str
] = {} This mapping will be looked up to choose an appropriate error message for history output if execution finished with errors. If no code found, then just “ERROR” string outputs.
- get_args(raw_args)
Returns list of processed arguments which will be substituted into execution command.
- Parameters
raw_args (
dict
) – argument name-value mapping which should be processed.- Return type
- get_env_vars(project_data)
Returns env variables which will be used in execution, project’s env variables by default.
- Parameters
project_data – proxy of the project instance, allows you to access it’s readonly properties, such as
config
vars
,env_vars
etc.- Return type
- get_execution_data(execution_dir, raw_args, project_data)
Returns tuple of execution command and env variables. This method will be called directly by executor before execution starts.
- Parameters
execution_dir (
pathlib.Path
) – path to execution directory in which project copy located. All additional files that should be generated (e.g. inventory file) must be placed here.raw_args (
dict
) – argument name-value mapping which should be processed.project_data – proxy of the project instance, allows you to access it’s readonly properties, such as
config
vars
,env_vars
etc.
- Return type
- get_pre_commands(raw_args)
This method will be called before execution. Returns list of commands which are needed to be executed before main execution command.
- Parameters
raw_args (
dict
) – dictionary with arguments received from API.- Return type
- get_serializer_class(exclude_fields=())
Returns serializer class which will be used to generate fields for arguments. Uses metaclass returned by
_get_serializer_metaclass
method.- Parameters
exclude_fields (
tuple
) – field names that should not be presented in serializer.- Return type
- get_verbose_level(raw_args)
Returns verbose level used for history output and logging. Should be taken from execution arguments (usually from
verbose
argument). This method will be called directly by executor.
- property name: str
Returns name of plugin, class name by default. Primarily used to generate an appropriate model name for OpenAPI schema.
- post_execute_hook(cmd, raw_args)
This method will be called after execution.
- Parameters
cmd (
typing.List
[str
]) – list of arguments which were used for execution.raw_args (
dict
) – dictionary with arguments received from API.
- Return type
- prepare_execution_dir(dir)
Gets execution directory with copied project. All files needed for execution (e.g. generated inventory file) should be here.
- Parameters
dir (
pathlib.Path
) – path to execution directory in which project copy located. All additional files that should be generated (e.g. inventory file) must be placed here.- Return type
-
serializer_fields:
typing.Mapping
[str
,rest_framework.fields.Field
] = {} Fields mapping used to generate serializer. By default returned by
_get_serializer_fields
method.
- class polemarch.plugins.inventory.base.BasePlugin(options)
Base inventory plugin class from which any other plugin should inherit. The plugin itself is an entity which, on the one hand provides appropriate fields for the API, and on the other hand, manages inventory state.
- Parameters
options –
settings.ini
options for this plugin.
- _get_serializer_fields()
Returns field name and field instance mapping used to generate fields for serializer used for working with state in API.
- Return type
typing.Mapping
[str
,rest_framework.fields.Field
]
- _get_serializer_import_fields()
Returns field name and field instance mapping used to generate fields for serializer used for import action.
- Return type
typing.Mapping
[str
,rest_framework.fields.Field
]
- _get_serializer_import_metaclass()
Returns serializer metaclass used to generate fields in serializer for import action.
- Return type
typing.Type
[typing.Type
[vstutils.api.serializers.BaseSerializer
]]
- _get_serializer_metaclass()
Returns serializer metaclass used to generate fields in serializer for working with state in API.
- Return type
typing.Type
[typing.Type
[vstutils.api.serializers.BaseSerializer
]]
-
defaults:
typing.Mapping
[str
,typing.Any
] = {} Field name and its default value mapping used to initialize inventory state. Only works if plugin is
state_managed
.
- classmethod get_raw_inventory(inventory_string)
Returns raw inventory string used to show it on history page.
- get_serializer_class()
Returns serializer class which will be used for working with state in API. Uses metaclass returned by
_get_serializer_metaclass
method.- Return type
- get_serializer_import_class()
Returns serializer class which will be used for import action. Uses metaclass returned by
_get_serializer_metaclass
method.- Return type
- classmethod import_inventory(instance, data)
Method which implements importing inventory from external source. Must be implemented if
supports_import
isTrue
.- Parameters
data (
dict
) – data received from user in API as a result of import action.instance (polemarch.main.models.hosts.Inventory) – created
Inventory
instance.
- render_inventory(instance, execution_dir)
Renders inventory into text file and puts it into
execution_dir
directory. Additional files may be returned by second argument as list (or empty list, if no any).- Parameters
instance (polemarch.main.models.hosts.Inventory) –
Inventory
instance.execution_dir (
pathlib.Path
) – path
- Return type
-
serializer_fields:
typing.Mapping
[str
,rest_framework.fields.Field
] = {} Fields mapping used to generate serializer for working with state in API. By default returned by
_get_serializer_fields
method.
-
serializer_import_fields:
typing.Mapping
[str
,rest_framework.fields.Field
] = {} Fields mapping used to generate serializer for import action. By default returned by
_get_serializer_import_fields
method.