扩展saltstack组件

一、扩展Grains

我们知道，grains可以帮助收集minion端的静态信息。在更为复杂的业务环境中，可以使用自定义的grains帮助收集真实需要的静态或者业务信息;

grains流程:  master端定义脚本--->脚本sync到minion端--->minion刷新grains--->获取新的grains值

举例说明:

[root@linux-node1 ~]# cat /srv/salt/base/_grains/example.py
#!/usr/bin/env python
# _*_ coding:utf-8 _*_
__author__ = 'Charles Chang'
def grains():
    local = {}
    test = {'key':'value','key1':'value1','key2':'value2'}
    local['list'] =[1,2,3,4]
    local['string']='str'
    local['dict'] =test
    return local

sync grains到minion端　　

salt '*' saltutil.sync_grains
linux-node2.oldboyedu.com:
    - grains.example
linux-node1.oldboyedu.com:
    - grains.example

在minion端查看grains脚本位置

[root@linux-node1 _grains]# salt '*' cmd.run ls /var/cache/salt/minion/extmods/grains
linux-node2.oldboyedu.com:
    example.py
    example.pyc
linux-node1.oldboyedu.com:
    example.py
    example.pyc

自定义grains如下

[root@linux-node1 ~]# salt '*' grains.item list string dict
linux-node2.oldboyedu.com:
    ----------
    dict:
        ----------
        key:
            value
        key1:
            value1
        key2:
            value2
    list:
        - 1
        - 2
        - 3
        - 4
    string:
        str
linux-node1.oldboyedu.com:
    ----------
    dict:
        ----------
        key:
            value
        key1:
            value1
        key2:
            value2
    list:
        - 1
        - 2
        - 3
        - 4
    string:
        str

　　

　　

二、扩展module

使用salt自带的模块dig查看

from __future__ import absolute_import

# Import salt libs
import salt.utils

# Import python libs
import logging
import re
import socket

log = logging.getLogger(__name__)

__virtualname__ = 'dig'

def __virtual__():
    '''
    Only load module if dig binary is present
    '''
    return True if salt.utils.which('dig') else False

def A(host, nameserver=None):
    '''
    Return the A record for ``host``.

    Always returns a list.

    CLI Example:

    .. code-block:: bash

        salt ns1 dig.A www.google.com
    '''
    dig = ['dig', '+short', str(host), 'A']

    if nameserver is not None:
        dig.append('@{0}'.format(nameserver))

    cmd = __salt__['cmd.run_all'](dig, python_shell=False)
    # In this case, 0 is not the same as False
    if cmd['retcode'] != 0:
        log.warn(
            'dig returned exit code \'{0}\'. Returning empty list as '
            'fallback.'.format(
                cmd['retcode']
            )
        )
        return []

    # make sure all entries are IPs
    return [x for x in cmd['stdout'].split('\n') if check_ip(x)]

　　

下面编辑一个puppet模块文件,用于批量检查puppet的状态：

 [root@linux-node1 _modules]# pwd
 /srv/salt/base/_modules
 [root@linux-node1 _modules]# cat puppet.py
 #!/usr/bin/env python
 # _*_ coding:utf-8 _*_
 from __future__ import absolute_import
 from salt.ext.six.moves.urllib.request import urlopen as _urlopen
 import salt.utils
 import salt.utils.decorators as decorators

 @decorators.memoize
 def __detect_os():
     return salt.utils.which('puppet')

 def __virtual__():
     if __detect_os():
         return True
     return False

 def setmaster(master='linux-node1.oldboyedu.com',config_file='/etc/puppet/puppet.conf'):
     check='grep server {0}'.format(config_file)
     outcheck = __salt__['cmd.run'](check)
     if outcheck:
         cmdline = 'sed -i "s/server = .*/server = {0}/g" {1}'.format(master,config_file)
         output = __salt__['cmd.run'](cmdline)
         return 'has change server to {0}'.format(master)
     else:
         cmdline = 'echo "   server = {0}" >> {1}'.format(master,config_file)
         output = __salt__['cmd.run'](cmdline)
         return 'has change server to {0} need restart the service'.format(master)

 def version():
     cmd = 'rpm -qf {0}'.format(__detect_os())
     output = __salt__['cmd.run'](cmd).splitlines()
     ret = output[0].split(': ')
     return ret[-1]

 def service(single=None):
     status = ('start','stop','status','restart','reload','force-reload','condrestart','once','genconfig')
     if single not in status:
         return 'puppet can not suppert this single'
     cmdline = 'systemctl {0} puppet.service'.format(single)
     output = __salt__['cmd.run'](cmdline)
     return output

 def master(config_file='/etc/puppet/puppet.conf'):
     cmdline= 'grep server' + '{0}'.format(config_file)
     output=__salt__['cmd.run'](cmdline,python_shell=False)
     if output:
         return output
     else:
         return 'puppet server not setting'

puppet.py

执行命令,将module同步到所有的minion端:

[root@linux-node1 ~]# salt '*' saltutil.sync_modules
linux-node1.oldboyedu.com:
    - modules.puppet
linux-node2.oldboyedu.com:
    - modules.puppet

同步结果:

[root@linux-node1 ~]# salt '*' cmd.run ls /var/cache/salt/minion/extmods/modules/
linux-node2.oldboyedu.com:
    puppet.py
    puppet.pyc
linux-node1.oldboyedu.com:
    puppet.py
    puppet.pyc

检测puppet版本:

[root@linux-node1 ~]# salt '*' puppet.version
linux-node2.oldboyedu.com:
    puppet-3.8.7-2.el7.noarch
linux-node1.oldboyedu.com:
    puppet-3.6.2-3.el7.noarch
[root@linux-node1 ~]#
[root@linux-node1 ~]# salt '*' puppet.service status
linux-node2.oldboyedu.com:
    * puppet.service - Puppet agent
       Loaded: loaded (/usr/lib/systemd/system/puppet.service; disabled; vendor preset: disabled)
       Active: inactive (dead)
linux-node1.oldboyedu.com:
    * puppet.service - Puppet agent
       Loaded: loaded (/usr/lib/systemd/system/puppet.service; disabled; vendor preset: disabled)
       Active: inactive (dead)
ERROR: Minions returned with non-zero exit code

[root@linux-node1 ~]# salt '*' puppet.master
linux-node2.oldboyedu.com:
    puppet server not setting
linux-node1.oldboyedu.com:
    puppet server not setting
ERROR: Minions returned with non-zero exit code

[root@linux-node1 ~]# salt '*' puppet.setmaster
linux-node2.oldboyedu.com:
    has change server to linux-node1.oldboyedu.com need restart the service
linux-node1.oldboyedu.com:
    has change server to linux-node1.oldboyedu.com need restart the service

[root@linux-node1 ~]# salt '*' puppet.master
linux-node2.oldboyedu.com:
    puppet server not setting
linux-node1.oldboyedu.com:
    puppet server not setting
ERROR: Minions returned with non-zero exit code

可以将平时有点繁琐的流程操作封装为saltstack module的方式,方便以后重复使用;

三、state扩展

state sls执行流程:sls的yaml格式使yaml_safe_load为python的字典数据类型--->state调用module操作

看官方的例子:

[root@linux-node1 states]# vim rabbitmq_user.py
# -*- coding: utf-8 -*-
'''
Manage RabbitMQ Users
=====================

Example:

.. code-block:: yaml

    rabbit_user:
      rabbitmq_user.present:
        - password: password
        - force: True
        - tags:
          - monitoring
          - user
        - perms:
          - '/':
            - '.*'
            - '.*'
            - '.*'
        - runas: rabbitmq
'''
from __future__ import absolute_import

# Import python libs
import logging

# Import salt libs
import salt.utils

log = logging.getLogger(__name__)

def __virtual__():
    '''
    Only load if RabbitMQ is installed.
    '''
    return salt.utils.which('rabbitmqctl') is not None

def absent(name,
           runas=None):
    '''
    Ensure the named user is absent

    name
        The name of the user to remove
    runas
        User to run the command
    '''
    ret = {'name': name, 'result': True, 'comment': '', 'changes': {}}

    user_exists = __salt__['rabbitmq.user_exists'](name, runas=runas)    #rabbitmq.user_exists是调用modules目录下rabbitmq.py文件中user_exists函数

    if not user_exists:
        ret['comment'] = 'User {0} is not present'.format(name)
    elif __opts__['test']:
        ret['result'] = None
        if user_exists:
            ret['comment'] = 'Removing user {0}'.format(name)
    else:
        result = __salt__['rabbitmq.delete_user'](name, runas=runas)
        if 'Error' in result:
            ret['result'] = False
            ret['comment'] = result['Error']
        elif 'Deleted' in result:
            ret['comment'] = 'Deleted'
            ret['changes'] = {'new': '', 'old': name}

    return ret

必须返回一个字典　

下面编写一个检测ansible.cfg文件的state，并使用sls进行调用:

[root@linux-node1 _states]# pwd
/srv/salt/base/_states
[root@linux-node1 _states]# cat ansible.py
#!/usr/bin/env python
# _*_ coding:utf-8 _*_
from __future__ import absolute_import
import salt.utils
import ConfigParser

def files(name='/etc/ansible/ansible.cfg ',
         inventory=None,
         forks=None,
         module_lang=None,
         host_key_checking=None,
         timeout=None
          ):
    ret = {
        'name':name,
        'changes':{},
        'result':True,
        'comment':''
        }
    all = {}
    file = __salt__['file.file_exists'](name)
    if __opts__['test']:              #是否在命令行加test=True
        ret['comment'] = 'some  default has changed'
        ret['result']=None
        return ret
    if file:
        cf = ConfigParser.ConfigParser()
        cf.read(name)
        if inventory:
            cf.set('defaults','inventory',inventory)
            mess = 'inventory change to {0} \n'.format(inventory)
        if forks:
            cf.set('defaults','forks',forks)
            mess +='fork changing to {0} \n'.format(forks)
        if module_lang:
            cf.set('defaults','module_lang',module_lang)
            mess += 'module_lang chnge to {0} \n'.format(module_lang)
        if host_key_checking:
            cf.set('defaults','host_key_checking',host_key_checking)
            mess +='host_key_checking to {0} \n'.format(host_key_checking)
        if timeout:
            cf.set('defaults','timeout',timeout)
            mess +='timout change to {0} \n'.format(timeout)
        cf.write(open(name,"w"))
        ret['result'] = True
        all['message']=mess
        ret['comment'] = 'some default has changed'
        ret['changes']=all
    else:
        ret['comment']= '{0} files not exists'.format(name)
        ret['result'] = False
    return ret

sls文件如下:

[root@linux-node1 base]# ls
ansible.sls  _states
[root@linux-node1 base]# cat ansible.sls
'/etc/ansible/ansible.cfg':
  ansible.files:
    - inventory: /etc/host
    - timeout: 88
    - forks: 8

同步到minion端:

[root@linux-node1 _states]# salt '*' saltutil.sync_states
linux-node1.oldboyedu.com:
    - states.ansible
linux-node2.oldboyedu.com:
    - states.ansible

[root@linux-node1 base]# salt '*' cmd.run ls /var/cache/salt/minion/extmods/states/
linux-node1.oldboyedu.com:
    ansible.py
    ansible.pyc
linux-node2.oldboyedu.com:
    ansible.py
    ansible.pyc

　　

测试:

[root@linux-node1 _states]# salt '*' state.sls ansible test=True
linux-node1.oldboyedu.com:
----------
          ID: /etc/ansible/ansible.cfg
    Function: ansible.files
      Result: None
     Comment: some  default has changed
     Started: 10:37:37.868769
    Duration: 11.797 ms
     Changes:   

Summary
------------
Succeeded: 1 (unchanged=1)
Failed:    0
------------
Total states run:     1
linux-node2.oldboyedu.com:
----------
          ID: /etc/ansible/ansible.cfg
    Function: ansible.files
      Result: None
     Comment: some  default has changed
     Started: 10:37:37.852572
    Duration: 6.358 ms
     Changes:   

Summary
------------
Succeeded: 1 (unchanged=1)
Failed:    0
------------
Total states run:     1

[root@linux-node1 _states]# salt '*' state.sls ansible
linux-node2.oldboyedu.com:
----------
          ID: /etc/ansible/ansible.cfg
    Function: ansible.files
      Result: False
     Comment: /etc/ansible/ansible.cfg files not exists
     Started: 10:39:21.038324
    Duration: 10.685 ms
     Changes:   

Summary
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
linux-node1.oldboyedu.com:
----------
          ID: /etc/ansible/ansible.cfg
    Function: ansible.files
      Result: True
     Comment: some default has changed
     Started: 10:39:21.082694
    Duration: 10.97 ms
     Changes:
              ----------
              message:
                  inventory change to /etc/host
                  fork changing to 8
                  timout change to 88 

Summary
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1

[root@linux-node1 _states]# salt '*' cmd.run "cat /etc/ansible/ansible.cfg|egrep '^inventory|^forks|^timeout'"
linux-node1.oldboyedu.com:
    inventory = /etc/host
    forks = 8
    timeout = 88
linux-node2.oldboyedu.com:
    cat: /etc/ansible/ansible.cfg: No such file or directory
ERROR: Minions returned with non-zero exit code

四、ext_pillar与ext_nodes

1、理解ext_pillar

先可以一个例子:

[root@linux-node1 ~]# vim /usr/lib/python2.7/site-packages/salt/pillar/hiera.py

# -*- coding: utf-8 -*-
'''
Use hiera data as a Pillar source
'''
from __future__ import absolute_import

# Import python libs
import logging

# Import salt libs
import salt.utils
from salt.ext.six import string_types

# Import third party libs
import yaml

# Set up logging
log = logging.getLogger(__name__)

def __virtual__():
    '''
    Only return if hiera is installed
    '''
    return 'hiera' if salt.utils.which('hiera') else False

def ext_pillar(minion_id,  # pylint: disable=W0613
               pillar,  # pylint: disable=W0613
               conf):
    '''
    Execute hiera and return the data
    '''
    cmd = 'hiera -c {0}'.format(conf)
    for key, val in __grains__.items():
        if isinstance(val, string_types):
            cmd += ' {0}={1!r}'.format(key, val)
    try:
        data = yaml.safe_load(__salt__['cmd.run'](cmd))
    except Exception:
        log.critical(
                'Hiera YAML data failed to parse from conf {0}'.format(conf)
                )
        return {}
    return data

pillar是通过yaml定义的，通过yaml_safe_load后就是python的数据类型了,也就是说从外部获取的pillar数据就是python里面的字典类型数据，salt就完成了对pillar数据的定义;目前官方已经支持24种从外部获取pillar数据的方式，当然可以自己定义从哪些地方获取pillar数据;

2、ext_nodes

在定义完pillar之后，需要在top.sls文件中指定pillar和minion之间的对于关系，也就是哪些minion会应用这些pillar,也就是说需要让top.sls支持动态的功能

　

五、saltstack git文件服务器