Skip to main content
BeeGFS on NetApp with E-Series Storage
简体中文版经机器翻译而成,仅供参考。如与英语版出现任何冲突,应以英语版为准。

为BeeGFS组件定义Ansible清单

贡献者

定义了常规Ansible清单结构后、为BeeGFS文件系统中的每个组件定义配置。

这些部署说明演示了如何部署文件系统、该文件系统由包括管理、元数据和存储服务在内的基础组件、包含元数据和存储服务的第二个组件以及仅存储的第三个组件组成。

这些步骤旨在显示一系列典型配置文件、您可以使用这些配置文件来配置NetApp BeeGFS组件、以满足整个BeeGFS文件系统的要求。

备注 在本节及后续章节中、根据需要进行调整、以构建表示要部署的BeeGFS文件系统的清单。特别是、请使用表示每个块或文件节点的Ansible主机名以及存储网络所需的IP寻址方案、以确保它可以扩展到BeeGFS文件节点和客户端的数量。

第1步:创建Ansible清单文件

步骤
  1. 创建新的`inventory.yml`文件、然后插入以下参数、根据需要替换`Esery_storage_systems`下的主机、以表示部署中的块节点。这些名称应与`host_vars/<filename>.yml`使用的名称相对应。

    # BeeGFS HA (High Availability) cluster inventory.
    all:
      children:
        # Ansible group representing all block nodes:
        eseries_storage_systems:
          hosts:
            netapp_01:
            netapp_02:
            netapp_03:
            netapp_04:
            netapp_05:
            netapp_06:
        # Ansible group representing all file nodes:
        ha_cluster:
          children:

    在后续各节中、您将在`ha_cluster`下创建其他Ansible组、这些组表示您要在集群中运行的BeeGFS服务。

第2步:为管理、元数据和存储组件配置清单

集群或基础构建块中的第一个组件必须包括BeeGFS管理服务以及元数据和存储服务:

步骤
  1. 在`inventory.yml`中、在`ha_cluster下填充以下参数:子项`:

          # beegfs_01/beegfs_02 HA Pair (mgmt/meta/storage building block):
            mgmt:
              hosts:
                beegfs_01:
                beegfs_02:
            meta_01:
              hosts:
                beegfs_01:
                beegfs_02:
            stor_01:
              hosts:
                beegfs_01:
                beegfs_02:
            meta_02:
              hosts:
                beegfs_01:
                beegfs_02:
            stor_02:
              hosts:
                beegfs_01:
                beegfs_02:
            meta_03:
              hosts:
                beegfs_01:
                beegfs_02:
            stor_03:
              hosts:
                beegfs_01:
                beegfs_02:
            meta_04:
              hosts:
                beegfs_01:
                beegfs_02:
            stor_04:
              hosts:
                beegfs_01:
                beegfs_02:
            meta_05:
              hosts:
                beegfs_02:
                beegfs_01:
            stor_05:
              hosts:
                beegfs_02:
                beegfs_01:
            meta_06:
              hosts:
                beegfs_02:
                beegfs_01:
            stor_06:
              hosts:
                beegfs_02:
                beegfs_01:
            meta_07:
              hosts:
                beegfs_02:
                beegfs_01:
            stor_07:
              hosts:
                beegfs_02:
                beegfs_01:
            meta_08:
              hosts:
                beegfs_02:
                beegfs_01:
            stor_08:
              hosts:
                beegfs_02:
                beegfs_01:
  2. 创建文件`group_vars/mgmt.yml`并包含以下内容:

    # mgmt - BeeGFS HA Management Resource Group
    # OPTIONAL: Override default BeeGFS management configuration:
    # beegfs_ha_beegfs_mgmtd_conf_resource_group_options:
    #  <beegfs-mgmt.conf:key>:<beegfs-mgmt.conf:value>
    floating_ips:
      - i1b: 100.127.101.0/16
      - i2b: 100.127.102.0/16
    beegfs_service: management
    beegfs_targets:
      netapp_01:
        eseries_storage_pool_configuration:
          - name: beegfs_m1_m2_m5_m6
            raid_level: raid1
            criteria_drive_count: 4
            common_volume_configuration:
              segment_size_kb:  128
            volumes:
              - size: 1
                owning_controller: A
  3. 在`group_vars/`下、使用以下模板为资源组`mETA_01`到`mETA_08`创建文件、然后参考下表填写每个服务的占位值:

    # meta_0X - BeeGFS HA Metadata Resource Group
    beegfs_ha_beegfs_meta_conf_resource_group_options:
      connMetaPortTCP: <PORT>
      connMetaPortUDP: <PORT>
      tuneBindToNumaZone: <NUMA ZONE>
    floating_ips:
      - <PREFERRED PORT:IP/SUBNET> # Example: i1b:192.168.120.1/16
      - <SECONDARY PORT:IP/SUBNET>
    beegfs_service: metadata
    beegfs_targets:
      <BLOCK NODE>:
        eseries_storage_pool_configuration:
          - name: <STORAGE POOL>
            raid_level: raid1
            criteria_drive_count: 4
            common_volume_configuration:
              segment_size_kb:  128
            volumes:
              - size: 21.25 # SEE NOTE BELOW!
                owning_controller: <OWNING CONTROLLER>
    备注 卷大小以整个存储池(也称为卷组)的百分比形式指定。NetApp强烈建议您在每个池中保留一些可用容量、以便为SSD过度配置留出空间(有关详细信息、请参见 "NetApp EF600阵列简介")。存储池`beegfs_m1_m2_m3_m3_m6`也会将池容量的1%分配给管理服务。因此、对于存储池中的元数据卷`beegfs_m1_m2_m5_m6`、如果使用1.92 TB或3.84 TB驱动器、请将此值设置为`21.25`;对于7.65 TB驱动器、请将此值设置为`22.25`;对于15.3 TB驱动器、请将此值设置为`23.75`。
    文件名 Port 浮动IP NUMA区域 块节点 存储池 所属控制器

    meta_01.yml

    8015

    i1b:100.127.101.1/16 i2b:100.127.102.1/16

    0

    netapp_01

    beegfs_m1_m2_m3_m3_m6

    meta_02.yml

    8025

    i2b:100.127.102.2/16 i1b:100.127.101.2/ 16

    0

    netapp_01

    beegfs_m1_m2_m3_m3_m6

    B

    meta_03.yml

    8035

    i3b:100.127.101.3/16 i4b:100.127.102.3/ 16

    1.

    netapp_02

    beegfs_m3_m4_m7_m8

    meta_04.yml

    8045

    i4b:100.127.102.4/16 i3b:100.127.101.4/ 16

    1.

    netapp_02

    beegfs_m3_m4_m7_m8

    B

    meta_05.yml

    8055

    i1b:100.127.101.5/16 i2b:100.127.102.5/ 16

    0

    netapp_01

    beegfs_m1_m2_m3_m3_m6

    meta_06.yml

    8065

    i2b:100.127.102.6/16 i1b:100.127.101.6/ 16

    0

    netapp_01

    beegfs_m1_m2_m3_m3_m6

    B

    meta_07.yml

    8075

    i3b:100.127.101.7/16 i4b:100.127.102.7/ 16

    1.

    netapp_02

    beegfs_m3_m4_m7_m8

    meta_08.yml

    8085

    i4b:100.127.102.8/16 i3b:100.127.101.8/ 16

    1.

    netapp_02

    beegfs_m3_m4_m7_m8

    B

  4. 在`group_vars/`下、使用以下模板为资源组`stor_01`到`stor_08`创建文件、然后参考以下示例填写每个服务的占位值:

    # stor_0X - BeeGFS HA Storage Resource Groupbeegfs_ha_beegfs_storage_conf_resource_group_options:
      connStoragePortTCP: <PORT>
      connStoragePortUDP: <PORT>
      tuneBindToNumaZone: <NUMA ZONE>
    floating_ips:
      - <PREFERRED PORT:IP/SUBNET>
      - <SECONDARY PORT:IP/SUBNET>
    beegfs_service: storage
    beegfs_targets:
      <BLOCK NODE>:
        eseries_storage_pool_configuration:
          - name: <STORAGE POOL>
            raid_level: raid6
            criteria_drive_count: 10
            common_volume_configuration:
              segment_size_kb: 512        volumes:
              - size: 21.50 # See note below!             owning_controller: <OWNING CONTROLLER>
              - size: 21.50            owning_controller: <OWNING CONTROLLER>
    备注 有关要使用的正确大小、请参见 "建议的存储池过度配置百分比"
    文件名 Port 浮动IP NUMA区域 块节点 存储池 所属控制器

    STOR_01.yml

    8013

    i1b:100.127.103.1/16 i2b:100.127.104.1/ 16

    0

    netapp_01

    beegfs_s1_s2

    STOR_02.yml

    8023

    i2b:100.127.104.2/16 i1b:100.127.103.2/ 16

    0

    netapp_01

    beegfs_s1_s2

    B

    STOR_03.yml

    8033

    i3b:100.127.103.3/16 i4b:100.127.104.3/ 16

    1.

    netapp_02

    beegfs_s3_s4.

    STOR_04.yml

    8043

    i4b:100.127.104.4/16 i3b:100.127.103.4/ 16

    1.

    netapp_02

    beegfs_s3_s4.

    B

    STOR_05.yml

    8053

    i1b:100.127.103.5/16 i2b:100.127.104.5/ 16

    0

    netapp_01

    beegfs_s5_s6

    STOR_06.yml

    8063

    i2b:100.127.104.6/16 i1b:100.127.103.6/ 16

    0

    netapp_01

    beegfs_s5_s6

    B

    STOR_07.yml

    8073.

    i3b:100.127.103.7/16 i4b:100.127.104.7/ 16

    1.

    netapp_02

    beegfs_s7_s8.

    STOR_08.yml

    8083.

    i4b:100.127.104.8/16 i3b:100.127.103.8/ 16

    1.

    netapp_02

    beegfs_s7_s8.

    B

第3步:为元数据+存储构建块配置清单

以下步骤介绍如何为BeeGFS元数据+存储构建块设置Ansible清单。

步骤
  1. 在`inventory.yml`中、在现有配置下填充以下参数:

            meta_09:
              hosts:
                beegfs_03:
                beegfs_04:
            stor_09:
              hosts:
                beegfs_03:
                beegfs_04:
            meta_10:
              hosts:
                beegfs_03:
                beegfs_04:
            stor_10:
              hosts:
                beegfs_03:
                beegfs_04:
            meta_11:
              hosts:
                beegfs_03:
                beegfs_04:
            stor_11:
              hosts:
                beegfs_03:
                beegfs_04:
            meta_12:
              hosts:
                beegfs_03:
                beegfs_04:
            stor_12:
              hosts:
                beegfs_03:
                beegfs_04:
            meta_13:
              hosts:
                beegfs_04:
                beegfs_03:
            stor_13:
              hosts:
                beegfs_04:
                beegfs_03:
            meta_14:
              hosts:
                beegfs_04:
                beegfs_03:
            stor_14:
              hosts:
                beegfs_04:
                beegfs_03:
            meta_15:
              hosts:
                beegfs_04:
                beegfs_03:
            stor_15:
              hosts:
                beegfs_04:
                beegfs_03:
            meta_16:
              hosts:
                beegfs_04:
                beegfs_03:
            stor_16:
              hosts:
                beegfs_04:
                beegfs_03:
  2. 在`group_vars/`下、使用以下模板为资源组`mETA_09`到`mETA_16`创建文件、然后参考以下示例填写每个服务的占位值:

    # meta_0X - BeeGFS HA Metadata Resource Group
    beegfs_ha_beegfs_meta_conf_resource_group_options:
      connMetaPortTCP: <PORT>
      connMetaPortUDP: <PORT>
      tuneBindToNumaZone: <NUMA ZONE>
    floating_ips:
      - <PREFERRED PORT:IP/SUBNET>
      - <SECONDARY PORT:IP/SUBNET>
    beegfs_service: metadata
    beegfs_targets:
      <BLOCK NODE>:
        eseries_storage_pool_configuration:
          - name: <STORAGE POOL>
            raid_level: raid1
            criteria_drive_count: 4
            common_volume_configuration:
              segment_size_kb: 128
            volumes:
              - size: 21.5 # SEE NOTE BELOW!
                owning_controller: <OWNING CONTROLLER>
    备注 有关要使用的正确大小、请参见 "建议的存储池过度配置百分比"
    文件名 Port 浮动IP NUMA区域 块节点 存储池 所属控制器

    meta_09.yml

    8015

    i1b:100.127.101.9/16 i2b:100.127.102.9/ 16

    0

    netapp_03

    Beegfs_M9_M10_M13_M14

    meta_10.yml

    8025

    i2b:100.127.102.10/16 i1b:100.127.101.10/ 16

    0

    netapp_03

    Beegfs_M9_M10_M13_M14

    B

    meta_11.yml

    8035

    i3b:100.127.2.1.1/16 i4b:100.127.102.11/16

    1.

    netapp_04

    Beegfs_M11_M12_M15_16

    meta_12.yml

    8045

    i4b:100.127.102.12/16 i3b:100.127.101.12/ 16

    1.

    netapp_04

    Beegfs_M11_M12_M15_16

    B

    meta_13.yml

    8055

    i1b:100.127.107.13/16 i2b:100.127.102.13/16

    0

    netapp_03

    Beegfs_M9_M10_M13_M14

    meta_14.yml

    8065

    i2b:100.127.102.14/16 i1b:100.127.101.14/ 16

    0

    netapp_03

    Beegfs_M9_M10_M13_M14

    B

    meta_15.yml

    8075

    i3b:100.127.101.15/16 i4b:100.127.102.15/ 16

    1.

    netapp_04

    Beegfs_M11_M12_M15_16

    meta_16.yml

    8085

    i4b:100.127.102.16/16 i3b:100.127.101.16/ 16

    1.

    netapp_04

    Beegfs_M11_M12_M15_16

    B

  3. 在`group_vars/下、`使用以下模板为资源组`stor_09`到`stor_16`创建文件、然后参考以下示例填写每个服务的占位值:

    # stor_0X - BeeGFS HA Storage Resource Group
    beegfs_ha_beegfs_storage_conf_resource_group_options:
      connStoragePortTCP: <PORT>
      connStoragePortUDP: <PORT>
      tuneBindToNumaZone: <NUMA ZONE>
    floating_ips:
      - <PREFERRED PORT:IP/SUBNET>
      - <SECONDARY PORT:IP/SUBNET>
    beegfs_service: storage
    beegfs_targets:
      <BLOCK NODE>:
        eseries_storage_pool_configuration:
          - name: <STORAGE POOL>
            raid_level: raid6
            criteria_drive_count: 10
            common_volume_configuration:
              segment_size_kb: 512        volumes:
              - size: 21.50 # See note below!
                owning_controller: <OWNING CONTROLLER>
              - size: 21.50            owning_controller: <OWNING CONTROLLER>
    备注 有关要使用的正确大小、请参见 "建议的存储池过度配置百分比"…​
    文件名 Port 浮动IP NUMA区域 块节点 存储池 所属控制器

    STOR_09.yml

    8013

    i1b:100.127.103.9/16 i2b:100.127.104.9/ 16

    0

    netapp_03

    beegfs_s9_s10

    STOR_10.yml

    8023

    i2b:100.127.104.10/16 i1b:100.127.103.10/ 16

    0

    netapp_03

    beegfs_s9_s10

    B

    STOR_11.yml

    8033

    i3b:100.127.103.11/16 i4b:100.127.104.11/ 16

    1.

    netapp_04

    beegfs_s11_s12

    STOR_12.yml

    8043

    i4b:100.127.104.12/16 i3b:100.127.103.12/ 16

    1.

    netapp_04

    beegfs_s11_s12

    B

    STOR_13.yml

    8053

    i1b:100.127.103.13/16 i2b:100.127.104.13/ 16

    0

    netapp_03

    beegfs_s13_s14

    STOR_14.yml

    8063

    i2b:100.127.104.14/16 i1b:100.127.103.14/ 16

    0

    netapp_03

    beegfs_s13_s14

    B

    STOR_15.yml

    8073.

    i3b:100.127.103.15/16 i4b:100.127.104.15/ 16

    1.

    netapp_04

    beegfs_s15_s16

    STOR_16.yml

    8083.

    i4b:100.127.104.16/16 i3b:100.127.103.16/ 16

    1.

    netapp_04

    beegfs_s15_s16

    B

第4步:为纯存储组件配置清单

以下步骤介绍如何为BeeGFS纯存储组件设置Ansible清单。设置元数据+存储与仅存储组件的配置之间的主要区别是、省略了所有元数据资源组、并将每个存储池的`Criteria_drive_count`从10更改为12。

步骤
  1. 在`inventory.yml`中、在现有配置下填充以下参数:

          # beegfs_05/beegfs_06 HA Pair (storage only building block):
            stor_17:
              hosts:
                beegfs_05:
                beegfs_06:
            stor_18:
              hosts:
                beegfs_05:
                beegfs_06:
            stor_19:
              hosts:
                beegfs_05:
                beegfs_06:
            stor_20:
              hosts:
                beegfs_05:
                beegfs_06:
            stor_21:
              hosts:
                beegfs_06:
                beegfs_05:
            stor_22:
              hosts:
                beegfs_06:
                beegfs_05:
            stor_23:
              hosts:
                beegfs_06:
                beegfs_05:
            stor_24:
              hosts:
                beegfs_06:
                beegfs_05:
  2. 在`group_vars/`下、使用以下模板为资源组`stor_17`到`stor_24`创建文件、然后参考以下示例填写每个服务的占位值:

    # stor_0X - BeeGFS HA Storage Resource Group
    beegfs_ha_beegfs_storage_conf_resource_group_options:
      connStoragePortTCP: <PORT>
      connStoragePortUDP: <PORT>
      tuneBindToNumaZone: <NUMA ZONE>
    floating_ips:
      - <PREFERRED PORT:IP/SUBNET>
      - <SECONDARY PORT:IP/SUBNET>
    beegfs_service: storage
    beegfs_targets:
      <BLOCK NODE>:
        eseries_storage_pool_configuration:
          - name: <STORAGE POOL>
            raid_level: raid6
            criteria_drive_count: 12
            common_volume_configuration:
              segment_size_kb: 512
            volumes:
              - size: 21.50 # See note below!
                owning_controller: <OWNING CONTROLLER>
              - size: 21.50
                owning_controller: <OWNING CONTROLLER>
    备注 有关要使用的正确大小、请参见 "建议的存储池过度配置百分比"
    文件名 Port 浮动IP NUMA区域 块节点 存储池 所属控制器

    STOR_17.yml

    8013

    i1b:100.127.103.17/16 i2b:100.127.104.17/ 16

    0

    netapp_05

    beegfs_s17_s18

    STOR_18.yml

    8023

    i2b:100.127.104.18/16 i1b:100.127.103.18/ 16

    0

    netapp_05

    beegfs_s17_s18

    B

    STOR_19.yml

    8033

    i3b:100.127.103.19/16 i4b:100.127.104.19/ 16

    1.

    netapp_06

    beegfs_s19_s20

    STOR_20.yml

    8043

    i4b:100.127.104.20/16 i3b:100.127.103.20/ 16

    1.

    netapp_06

    beegfs_s19_s20

    B

    STOR_21.yml

    8053

    i1b:100.127.103.21/16 i2b:100.127.104.21/ 16

    0

    netapp_05

    Beegfs_S21_S22

    STOR_22.yml

    8063

    i2b:100.127.104.22/16 i1b:100.127.103.22/ 16

    0

    netapp_05

    Beegfs_S21_S22

    B

    STOR_23.yml

    8073.

    i3b:100.127.103.23/16 i4b:100.127.104.23/ 16

    1.

    netapp_06

    beegfs_s23_s24

    STOR_24.yml

    8083.

    i4b:100.127.104.24/16 i3b:100.127.103.24/ 16

    1.

    netapp_06

    beegfs_s23_s24

    B