Debian GNU/LinuxベースのNOSであるCumulus LinuxとSONiCの設定方法と設定ファイルについて簡単にまとめてみた。(というより晒しただけ)
製品版ではなくJenkinsから落としてくるSONiCのBGP設定について腑に落ちない所が有り、SONiCを再度デプロイから始め、Cumulus LinuxをBGPピアとしたeBGPの設定をもう一度試行。コマンド投入で設定可能な内容と設定ファイルの編集が必要な内容を整理してみた。
※今回の確認対象はBGPの設定に関わるファイルのみであり、VXLANやACLやQoSなど他の設定のファイルに関しては触れない事を予め断っておく。
当方の環境
SONiCはJenkinsからダウンロード可能なもの(開発版)、Cumulus Linuxは製品版の仮想アプライアンス版(Cumulus VX)を使用。VirtualBoxにデプロイした。
SONIC：
$ show version SONiC Software Version: SONiC.202012.35-84a091d9 Distribution: Debian 10.8 Kernel: 4.19.0-12-2-amd64 Build commit: 84a091d9 Build date: Fri Mar 5 06:47:10 UTC 2021 Built by: johnar@jenkins-worker-8
Cumulus VX：
$ net show version NCLU_VERSION=1.0-cl4.2.1u1 DISTRIB_ID="Cumulus Linux" DISTRIB_RELEASE=4.2.1 DISTRIB_DESCRIPTION="Cumulus Linux 4.2.1"
構成
今回の構成は以下の図のとおり。
IP Clos(っぽい)構成は本来であればLeafとSpineとの間の物理リンクは1本づつである。
今回は1台のSONiCと1台のCumulus VXとの間でeBGPで接続させた。
尚、SONiCはデプロイ直後には既にBGPやIPアドレスの設定が投入済みにつき設定変更箇所は最低限に留め、対向先のCumulusはSONiCに合わせる形で設定投入した。
Cumulus Linux (Cumulus VX)の場合
インタフェースやシステム周りはNCLUコマンドで設定を投入し、net commitコマンドで設定内容を保存。
ルーティングはvtyshモードに入りCisco IOSライクなコマンドで設定を投入し、write memoryコマンドで設定内容を保存。
製品版の仮想アプライアンス版という事も有ってか、検証環境の範囲内ではあるものの設定後はヘンテコな動作は見せず動作が安定している。
SONiCの場合
Jenkinsから落としてきたイメージファイルは既にBGPやルートマップやIPアドレス等の設定が入っている状態である為、自宅環境にてBGPの動作検証をする程度であれば、最初から設定投入をする事は殆ど無いだろう。
インタフェースやスタティックルートやホスト名等の設定は、sudo configコマンドで投入する。設定内容はsudo config saveコマンドで保存可能。
BGPなどダイナミックルーティングの設定はvtyshモードでCisco IOSライクなコマンドで投入する。
write memoryコマンドで設定内容の保存が可能だが、当方の環境ではvtyshモードのBGP設定が/etc/sonic/config_db.jsonには反映されずbgpd.confには反映された。
/etc/sonic/config_db.jsonファイルとbgpd.confファイルの立ち位置はどう違うのか調べていくと、以下の資料にたどり着いた。
https://speakerdeck.com/imasaruoki/sonicdeshe-ding-surufrrouting SONiCで設定するFRRouting
上記資料の9シート目の記載内容によると、設定内容が/etc/sonic/config_db.jsonにマージされるのではなく、Config DBの内容とvtyshモードで設定した内容(bgpd.confなど)が実行中のコンフィグレーションになる事が分かった。
①/etc/sonic/config_db.jsonファイルの内容をConfig DBに書き込み。
②bgpcfgdがConfig DBの以下のデータを読み込む。
・bgp_asn
・neighbor asn
・name
・admin status
③vtyshモードで投入した設定内容をbgpdに読み込ませる。
④上記の②と③の内容が、実行中のコンフィグレーションとなる。
Ciscoのスイッチで言うと②＋③の内容がrunning-configという事か。
ところで、BGPの設定投入後にwrite memoryコマンドを実行し設定内容をシステムに反映させようとすると...
"Note: this version of vtysh never writes vtysh.conf"と怒られた。
# write memory Note: this version of vtysh never writes vtysh.conf Building Configuration... Configuration saved to /etc/frr/zebra.conf Configuration saved to /etc/frr/bgpd.conf Configuration saved to /etc/frr/staticd.conf #
"Configuration saved to"で示されていたディレクトリは存在せずzebra.confとbgpd.confとstaticd.confファイルのパスは出力内容とは異なっており、/etc/frr/ディレクトリ内ではなく/etc/sonic/frr/ディレクトリ内に有った。
$ ls /etc/frr ls: cannot access '/etc/frr': No such file or directory $ $ sudo ls /etc/sonic/frr/ bgpd.conf bgpd.conf.sav staticd.conf staticd.conf.sav vtysh.conf zebra.conf zebra.conf.sav $
Cumulus Linuxの設定内容
以下は/etc/network/interfacesファイルの内容。
NCLUコマンドで投入したインタフェース周りの設定が書かれている。
$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*.intf
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
address 10.1.1.98/32
# The primary network interface
auto eth0
iface eth0
address 192.168.3.250/24
vrf mgmt
auto swp1
iface swp1
address 10.0.0.1/31
alias DEV=jpmtkvmsw99 IF=Eth0
mtu 9100
auto swp2
iface swp2
address 10.0.0.3/31
alias DEV=jpmtkvmsw99 IF=Eth1
mtu 9100
auto swp3
iface swp3
address 10.0.0.5/31
alias DEV=jpmtkvmsw99 IF=Eth2
mtu 9100
auto swp4
iface swp4
address 10.0.0.7/31
alias DEV=jpmtkvmsw99 IF=Eth3
mtu 9100
auto swp5
iface swp5
address 10.0.0.9/31
alias DEV=jpmtkvmsw99 IF=Eth4
mtu 9100
auto mgmt
iface mgmt
address 127.0.0.1/8
address ::1/128
vrf-table auto
$
以下は/etc/frr/frr.confファイルの内容。
vtyshモードで設定したルーティングの設定が書かれているが、NCLUとvtyshとの間で完全に棲み分けしているという訳ではないようで、NCLUで設定したhostnameは当ファイル内に書かれている。
$ sudo cat /etc/frr/frr.conf [sudo] password for cumulus: frr version 7.4+cl4.2.1u1 frr defaults datacenter hostname jpmtkvmsw98 log syslog informational service integrated-vtysh-config router bgp 65200 neighbor 10.0.0.0 remote-as 65100 neighbor 10.0.0.2 remote-as 65100 neighbor 10.0.0.4 remote-as 65100 neighbor 10.0.0.6 remote-as 65100 neighbor 10.0.0.8 remote-as 65100 address-family ipv4 unicast network 10.1.1.98/32 exit-address-family line vty $
SONiCの設定内容
以下は/etc/sonic/config_db.jsonファイルの内容。
"BGP_NEIGHBOR"の項目にてBGPネイバを修正すべく/etc/sonic/config_db.jsonファイルを直接編集した。
BGPネイバのCumulus Linuxに合わせてAS番号("asn"の項目)を修正、"name"の項目はCumulus Linuxのホスト名とインタフェース名に修正。
また、ずらーりずらりと並んでいたBGPネイバ設定の数はCumulus Linux側に合わせて5つまでとし、6つ目以降は削除した。
"BGP_NEIGHBOR"の項目以外の項目は編集しておらず、そのまま残している。
$ sudo cat /etc/sonic/config_db.json
{
"BGP_NEIGHBOR": {
"10.0.0.1": {
"asn": "65200",
"holdtime": "180",
"keepalive": "60",
"local_addr": "10.0.0.0",
"name": "jpmtkvmsw98_swp1",
"nhopself": "0",
"rrclient": "0"
},
"10.0.0.3": {
"asn": "65200",
"holdtime": "180",
"keepalive": "60",
"local_addr": "10.0.0.2",
"name": "jpmtkvmsw98_swp2",
"nhopself": "0",
"rrclient": "0"
},
"10.0.0.5": {
"asn": "65200",
"holdtime": "180",
"keepalive": "60",
"local_addr": "10.0.0.4",
"name": "jpmtkvmsw98_swp3",
"nhopself": "0",
"rrclient": "0"
},
"10.0.0.7": {
"asn": "65200",
"holdtime": "180",
"keepalive": "60",
"local_addr": "10.0.0.6",
"name": "jpmtkvmsw98_swp4",
"nhopself": "0",
"rrclient": "0"
},
"10.0.0.9": {
"asn": "65200",
"holdtime": "180",
"keepalive": "60",
"local_addr": "10.0.0.8",
"name": "jpmtkvmsw98_swp5",
"nhopself": "0",
"rrclient": "0"
}
},
"CRM": {
"Config": {
"acl_counter_high_threshold": "85",
"acl_counter_low_threshold": "70",
"acl_counter_threshold_type": "percentage",
"acl_entry_high_threshold": "85",
"acl_entry_low_threshold": "70",
"acl_entry_threshold_type": "percentage",
"acl_group_high_threshold": "85",
"acl_group_low_threshold": "70",
"acl_group_threshold_type": "percentage",
"acl_table_high_threshold": "85",
"acl_table_low_threshold": "70",
"acl_table_threshold_type": "percentage",
"dnat_entry_high_threshold": "85",
"dnat_entry_low_threshold": "70",
"dnat_entry_threshold_type": "percentage",
"fdb_entry_high_threshold": "85",
"fdb_entry_low_threshold": "70",
"fdb_entry_threshold_type": "percentage",
"ipmc_entry_high_threshold": "85",
"ipmc_entry_low_threshold": "70",
"ipmc_entry_threshold_type": "percentage",
"ipv4_neighbor_high_threshold": "85",
"ipv4_neighbor_low_threshold": "70",
"ipv4_neighbor_threshold_type": "percentage",
"ipv4_nexthop_high_threshold": "85",
"ipv4_nexthop_low_threshold": "70",
"ipv4_nexthop_threshold_type": "percentage",
"ipv4_route_high_threshold": "85",
"ipv4_route_low_threshold": "70",
"ipv4_route_threshold_type": "percentage",
"ipv6_neighbor_high_threshold": "85",
"ipv6_neighbor_low_threshold": "70",
"ipv6_neighbor_threshold_type": "percentage",
"ipv6_nexthop_high_threshold": "85",
"ipv6_nexthop_low_threshold": "70",
"ipv6_nexthop_threshold_type": "percentage",
"ipv6_route_high_threshold": "85",
"ipv6_route_low_threshold": "70",
"ipv6_route_threshold_type": "percentage",
"nexthop_group_high_threshold": "85",
"nexthop_group_low_threshold": "70",
"nexthop_group_member_high_threshold": "85",
"nexthop_group_member_low_threshold": "70",
"nexthop_group_member_threshold_type": "percentage",
"nexthop_group_threshold_type": "percentage",
"polling_interval": "300",
"snat_entry_high_threshold": "85",
"snat_entry_low_threshold": "70",
"snat_entry_threshold_type": "percentage"
}
},
"DEVICE_METADATA": {
"localhost": {
"bgp_asn": "65100",
"buffer_model": "traditional",
"default_bgp_status": "up",
"default_pfcwd_status": "disable",
"hostname": "jpmtkvmsw99",
"hwsku": "Force10-S6000",
"mac": "52:54:00:12:34:56",
"platform": "x86_64-kvm_x86_64-r0",
"type": "LeafRouter"
}
},
"FEATURE": {
"bgp": {
"auto_restart": "enabled",
"has_global_scope": "False",
"has_per_asic_scope": "True",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"database": {
"auto_restart": "always_enabled",
"has_global_scope": "True",
"has_per_asic_scope": "True",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "always_enabled"
},
"dhcp_relay": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"gbsyncd": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"lldp": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "True",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"mgmt-framework": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "True",
"high_mem_alert": "disabled",
"state": "enabled"
},
"nat": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "disabled"
},
"pmon": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"radv": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"sflow": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "disabled"
},
"snmp": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "True",
"high_mem_alert": "disabled",
"state": "enabled"
},
"swss": {
"auto_restart": "enabled",
"has_global_scope": "False",
"has_per_asic_scope": "True",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"syncd": {
"auto_restart": "enabled",
"has_global_scope": "False",
"has_per_asic_scope": "True",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"teamd": {
"auto_restart": "enabled",
"has_global_scope": "False",
"has_per_asic_scope": "True",
"has_timer": "False",
"high_mem_alert": "disabled",
"state": "enabled"
},
"telemetry": {
"auto_restart": "enabled",
"has_global_scope": "True",
"has_per_asic_scope": "False",
"has_timer": "True",
"high_mem_alert": "disabled",
"state": "enabled"
}
},
"FLEX_COUNTER_TABLE": {
"BUFFER_POOL_WATERMARK": {
"FLEX_COUNTER_STATUS": "enable"
},
"PFCWD": {
"FLEX_COUNTER_STATUS": "enable"
},
"PG_DROP": {
"FLEX_COUNTER_STATUS": "enable"
},
"PG_WATERMARK": {
"FLEX_COUNTER_STATUS": "enable"
},
"PORT": {
"FLEX_COUNTER_STATUS": "enable"
},
"PORT_BUFFER_DROP": {
"FLEX_COUNTER_STATUS": "enable"
},
"QUEUE": {
"FLEX_COUNTER_STATUS": "enable"
},
"QUEUE_WATERMARK": {
"FLEX_COUNTER_STATUS": "enable"
},
"RIF": {
"FLEX_COUNTER_STATUS": "enable"
}
},
"INTERFACE": {
"Ethernet0": {},
"Ethernet100": {},
"Ethernet104": {},
"Ethernet108": {},
"Ethernet112": {},
"Ethernet116": {},
"Ethernet12": {},
"Ethernet120": {},
"Ethernet124": {},
"Ethernet16": {},
"Ethernet20": {},
"Ethernet24": {},
"Ethernet28": {},
"Ethernet32": {},
"Ethernet36": {},
"Ethernet4": {},
"Ethernet40": {},
"Ethernet44": {},
"Ethernet48": {},
"Ethernet52": {},
"Ethernet56": {},
"Ethernet60": {},
"Ethernet64": {},
"Ethernet68": {},
"Ethernet72": {},
"Ethernet76": {},
"Ethernet8": {},
"Ethernet80": {},
"Ethernet84": {},
"Ethernet88": {},
"Ethernet92": {},
"Ethernet96": {},
"Ethernet0|10.0.0.0/31": {},
"Ethernet100|10.0.0.50/31": {},
"Ethernet104|10.0.0.52/31": {},
"Ethernet108|10.0.0.54/31": {},
"Ethernet112|10.0.0.56/31": {},
"Ethernet116|10.0.0.58/31": {},
"Ethernet120|10.0.0.60/31": {},
"Ethernet124|10.0.0.62/31": {},
"Ethernet12|10.0.0.6/31": {},
"Ethernet16|10.0.0.8/31": {},
"Ethernet20|10.0.0.10/31": {},
"Ethernet24|10.0.0.12/31": {},
"Ethernet28|10.0.0.14/31": {},
"Ethernet32|10.0.0.16/31": {},
"Ethernet36|10.0.0.18/31": {},
"Ethernet40|10.0.0.20/31": {},
"Ethernet44|10.0.0.22/31": {},
"Ethernet48|10.0.0.24/31": {},
"Ethernet4|10.0.0.2/31": {},
"Ethernet52|10.0.0.26/31": {},
"Ethernet56|10.0.0.28/31": {},
"Ethernet60|10.0.0.30/31": {},
"Ethernet64|10.0.0.32/31": {},
"Ethernet68|10.0.0.34/31": {},
"Ethernet72|10.0.0.36/31": {},
"Ethernet76|10.0.0.38/31": {},
"Ethernet80|10.0.0.40/31": {},
"Ethernet84|10.0.0.42/31": {},
"Ethernet88|10.0.0.44/31": {},
"Ethernet8|10.0.0.4/31": {},
"Ethernet92|10.0.0.46/31": {},
"Ethernet96|10.0.0.48/31": {}
},
"KDUMP": {
"config": {
"enabled": "false",
"memory": "0M-2G:256M,2G-4G:320M,4G-8G:384M,8G-:448M",
"num_dumps": "3"
}
},
"LOOPBACK_INTERFACE": {
"Loopback0": {},
"Loopback0|10.1.1.99/32": {}
},
"MGMT_INTERFACE": {
"eth0|192.168.3.254/24": {
"gwaddr": "192.168.3.1"
}
},
"PORT": {
"Ethernet0": {
"admin_status": "up",
"alias": "fortyGigE0/0",
"index": "0",
"lanes": "25,26,27,28",
"mtu": "9100",
"speed": "40000"
},
"Ethernet100": {
"admin_status": "up",
"alias": "fortyGigE0/100",
"index": "25",
"lanes": "121,122,123,124",
"mtu": "9100",
"speed": "40000"
},
"Ethernet104": {
"admin_status": "up",
"alias": "fortyGigE0/104",
"index": "26",
"lanes": "81,82,83,84",
"mtu": "9100",
"speed": "40000"
},
"Ethernet108": {
"admin_status": "up",
"alias": "fortyGigE0/108",
"index": "27",
"lanes": "85,86,87,88",
"mtu": "9100",
"speed": "40000"
},
"Ethernet112": {
"admin_status": "up",
"alias": "fortyGigE0/112",
"index": "28",
"lanes": "93,94,95,96",
"mtu": "9100",
"speed": "40000"
},
"Ethernet116": {
"admin_status": "up",
"alias": "fortyGigE0/116",
"index": "29",
"lanes": "89,90,91,92",
"mtu": "9100",
"speed": "40000"
},
"Ethernet12": {
"admin_status": "up",
"alias": "fortyGigE0/12",
"index": "3",
"lanes": "37,38,39,40",
"mtu": "9100",
"speed": "40000"
},
"Ethernet120": {
"admin_status": "up",
"alias": "fortyGigE0/120",
"index": "30",
"lanes": "101,102,103,104",
"mtu": "9100",
"speed": "40000"
},
"Ethernet124": {
"admin_status": "up",
"alias": "fortyGigE0/124",
"index": "31",
"lanes": "97,98,99,100",
"mtu": "9100",
"speed": "40000"
},
"Ethernet16": {
"admin_status": "up",
"alias": "fortyGigE0/16",
"index": "4",
"lanes": "45,46,47,48",
"mtu": "9100",
"speed": "40000"
},
"Ethernet20": {
"admin_status": "up",
"alias": "fortyGigE0/20",
"index": "5",
"lanes": "41,42,43,44",
"mtu": "9100",
"speed": "40000"
},
"Ethernet24": {
"admin_status": "up",
"alias": "fortyGigE0/24",
"index": "6",
"lanes": "1,2,3,4",
"mtu": "9100",
"speed": "40000"
},
"Ethernet28": {
"admin_status": "up",
"alias": "fortyGigE0/28",
"index": "7",
"lanes": "5,6,7,8",
"mtu": "9100",
"speed": "40000"
},
"Ethernet32": {
"admin_status": "up",
"alias": "fortyGigE0/32",
"index": "8",
"lanes": "13,14,15,16",
"mtu": "9100",
"speed": "40000"
},
"Ethernet36": {
"admin_status": "up",
"alias": "fortyGigE0/36",
"index": "9",
"lanes": "9,10,11,12",
"mtu": "9100",
"speed": "40000"
},
"Ethernet4": {
"admin_status": "up",
"alias": "fortyGigE0/4",
"index": "1",
"lanes": "29,30,31,32",
"mtu": "9100",
"speed": "40000"
},
"Ethernet40": {
"admin_status": "up",
"alias": "fortyGigE0/40",
"index": "10",
"lanes": "17,18,19,20",
"mtu": "9100",
"speed": "40000"
},
"Ethernet44": {
"admin_status": "up",
"alias": "fortyGigE0/44",
"index": "11",
"lanes": "21,22,23,24",
"mtu": "9100",
"speed": "40000"
},
"Ethernet48": {
"admin_status": "up",
"alias": "fortyGigE0/48",
"index": "12",
"lanes": "53,54,55,56",
"mtu": "9100",
"speed": "40000"
},
"Ethernet52": {
"admin_status": "up",
"alias": "fortyGigE0/52",
"index": "13",
"lanes": "49,50,51,52",
"mtu": "9100",
"speed": "40000"
},
"Ethernet56": {
"admin_status": "up",
"alias": "fortyGigE0/56",
"index": "14",
"lanes": "57,58,59,60",
"mtu": "9100",
"speed": "40000"
},
"Ethernet60": {
"admin_status": "up",
"alias": "fortyGigE0/60",
"index": "15",
"lanes": "61,62,63,64",
"mtu": "9100",
"speed": "40000"
},
"Ethernet64": {
"admin_status": "up",
"alias": "fortyGigE0/64",
"index": "16",
"lanes": "69,70,71,72",
"mtu": "9100",
"speed": "40000"
},
"Ethernet68": {
"admin_status": "up",
"alias": "fortyGigE0/68",
"index": "17",
"lanes": "65,66,67,68",
"mtu": "9100",
"speed": "40000"
},
"Ethernet72": {
"admin_status": "up",
"alias": "fortyGigE0/72",
"index": "18",
"lanes": "73,74,75,76",
"mtu": "9100",
"speed": "40000"
},
"Ethernet76": {
"admin_status": "up",
"alias": "fortyGigE0/76",
"index": "19",
"lanes": "77,78,79,80",
"mtu": "9100",
"speed": "40000"
},
"Ethernet8": {
"admin_status": "up",
"alias": "fortyGigE0/8",
"index": "2",
"lanes": "33,34,35,36",
"mtu": "9100",
"speed": "40000"
},
"Ethernet80": {
"admin_status": "up",
"alias": "fortyGigE0/80",
"index": "20",
"lanes": "109,110,111,112",
"mtu": "9100",
"speed": "40000"
},
"Ethernet84": {
"admin_status": "up",
"alias": "fortyGigE0/84",
"index": "21",
"lanes": "105,106,107,108",
"mtu": "9100",
"speed": "40000"
},
"Ethernet88": {
"admin_status": "up",
"alias": "fortyGigE0/88",
"index": "22",
"lanes": "113,114,115,116",
"mtu": "9100",
"speed": "40000"
},
"Ethernet92": {
"admin_status": "up",
"alias": "fortyGigE0/92",
"index": "23",
"lanes": "117,118,119,120",
"mtu": "9100",
"speed": "40000"
},
"Ethernet96": {
"admin_status": "up",
"alias": "fortyGigE0/96",
"index": "24",
"lanes": "125,126,127,128",
"mtu": "9100",
"speed": "40000"
}
},
"SNMP": {
"LOCATION": {
"Location": "public"
}
},
"SNMP_COMMUNITY": {
"public": {
"TYPE": "RO"
}
},
"VERSIONS": {
"DATABASE": {
"VERSION": "version_2_0_0"
}
}
}
$
以下は/etc/network/interfacesファイルの内容。
"Managed by SONiC Config Engine DO NOT EDIT!"の記載が有るとおり、このファイルは直接編集していない。
$ sudo cat /etc/network/interfaces
#
# =============== Managed by SONiC Config Engine DO NOT EDIT! ===============
# generated from /usr/share/sonic/templates/interfaces.j2 using sonic-cfggen
# file: /etc/network/interfaces
#
# The loopback network interface
auto lo
iface lo inet loopback
address 127.0.0.1
netmask 255.255.0.0
post-up ip addr del 127.0.0.1/8 dev lo
# The management network interface
auto eth0
iface eth0 inet static
address 192.168.3.254
netmask 255.255.255.0
network 192.168.3.0
broadcast 192.168.3.255
########## management network policy routing rules
# management port up rules
up ip -4 route add default via 192.168.3.1 dev eth0 table default metric 201
up ip -4 route add 192.168.3.0/24 dev eth0 table default
up ip -4 rule add pref 32765 from 192.168.3.254/32 table default
# management port down rules
pre-down ip -4 route delete default via 192.168.3.1 dev eth0 table default
pre-down ip -4 route delete 192.168.3.0/24 dev eth0 table default
pre-down ip -4 rule delete pref 32765 from 192.168.3.254/32 table default
#
source /etc/network/interfaces.d/*
#
$以下は/etc/sonic/frr/zebra.confファイルの内容。
frrバージョンやホスト名、パスワード、ログの設定等はzebra.confファイルおよびbgpd.conf、staticd.confの各ファイルにて同じである。
write memoryコマンドで設定内容が反映される為、当ファイルは直接編集していない。
$ sudo cat /etc/sonic/frr/zebra.conf ! ! Zebra configuration saved from vty ! 2021/03/20 02:00:24 ! frr version 7.5-sonic frr defaults traditional ! hostname jpmtkvmsw99 password zebra enable password zebra log syslog informational log facility local4 ! ! ! ip nht resolve-via-default ip protocol bgp route-map RM_SET_SRC ! ! route-map ALLOW_LIST_DEPLOYMENT_ID_0_V4 permit 65535 ! route-map ALLOW_LIST_DEPLOYMENT_ID_0_V6 permit 65535 ! route-map FROM_BGP_PEER_V4 permit 10 call ALLOW_LIST_DEPLOYMENT_ID_0_V4 on-match next ! route-map FROM_BGP_PEER_V4 permit 11 ! route-map FROM_BGP_PEER_V4 permit 100 ! route-map FROM_BGP_PEER_V6 permit 10 call ALLOW_LIST_DEPLOYMENT_ID_0_V6 on-match next ! route-map FROM_BGP_PEER_V6 permit 11 ! route-map FROM_BGP_PEER_V6 permit 1 on-match next ! route-map FROM_BGP_PEER_V6 permit 100 ! route-map TO_BGP_PEER_V4 permit 100 ! route-map TO_BGP_PEER_V6 permit 100 ! route-map RM_SET_SRC permit 10 set src 10.1.1.99 ! ! ! ! ! line vty ! $
以下は/etc/sonic/frr/bgpd.confファイルの内容。
ファイル名のとおりBGPの設定が保存されているファイルのようで、BGPの設定以外は、zebra.confファイルおよびstaticd.confファイルと同じである。
write memoryコマンドで設定内容が反映される為、当ファイルは直接編集していない。
BGPネイバの設定内容をよく見ると、ネイバの数やremote-as、descriptionの設定は/etc/sonic/config_db.jsonの内容と同じである事が分かる。
...という事は、BGPネイバの設定は/etc/sonic/config_db.jsonファイルを直接編集、BGPの他の設定やルートマップの設定等はvtyshモードで設定...という事なのだろうか。
$ sudo cat /etc/sonic/frr/bgpd.conf ! ! Zebra configuration saved from vty ! 2021/03/20 02:00:24 ! frr version 7.5-sonic frr defaults traditional ! hostname jpmtkvmsw99 password zebra enable password zebra log syslog informational log facility local4 ! ! ! router bgp 65100 bgp router-id 10.1.1.99 bgp log-neighbor-changes no bgp ebgp-requires-policy no bgp default ipv4-unicast bgp graceful-restart restart-time 240 bgp graceful-restart bgp graceful-restart preserve-fw-state bgp bestpath as-path multipath-relax neighbor PEER_V4 peer-group neighbor PEER_V6 peer-group neighbor 10.0.0.1 remote-as 65200 neighbor 10.0.0.1 peer-group PEER_V4 neighbor 10.0.0.1 description jpmtkvmsw98_swp1 neighbor 10.0.0.3 remote-as 65200 neighbor 10.0.0.3 peer-group PEER_V4 neighbor 10.0.0.3 description jpmtkvmsw98_swp2 neighbor 10.0.0.5 remote-as 65200 neighbor 10.0.0.5 peer-group PEER_V4 neighbor 10.0.0.5 description jpmtkvmsw98_swp3 neighbor 10.0.0.7 remote-as 65200 neighbor 10.0.0.7 peer-group PEER_V4 neighbor 10.0.0.7 description jpmtkvmsw98_swp4 neighbor 10.0.0.9 remote-as 65200 neighbor 10.0.0.9 peer-group PEER_V4 neighbor 10.0.0.9 description jpmtkvmsw98_swp5 ! address-family ipv4 unicast network 10.1.1.99/32 neighbor PEER_V4 soft-reconfiguration inbound neighbor PEER_V4 route-map FROM_BGP_PEER_V4 in neighbor PEER_V4 route-map TO_BGP_PEER_V4 out neighbor 10.0.0.1 activate neighbor 10.0.0.3 activate neighbor 10.0.0.5 activate neighbor 10.0.0.7 activate neighbor 10.0.0.9 activate maximum-paths 64 exit-address-family ! address-family ipv6 unicast neighbor PEER_V6 soft-reconfiguration inbound neighbor PEER_V6 route-map FROM_BGP_PEER_V6 in neighbor PEER_V6 route-map TO_BGP_PEER_V6 out maximum-paths 64 exit-address-family ! ! ip prefix-list PL_LoopbackV4 seq 5 permit 10.1.1.99/32 ! bgp community-list standard allow_list_default_community seq 5 permit no-export bgp community-list standard allow_list_default_community seq 10 permit 5060:12345 ! route-map ALLOW_LIST_DEPLOYMENT_ID_0_V4 permit 65535 set community 5060:12345 additive ! route-map ALLOW_LIST_DEPLOYMENT_ID_0_V6 permit 65535 set community 5060:12345 additive ! route-map FROM_BGP_PEER_V4 permit 10 call ALLOW_LIST_DEPLOYMENT_ID_0_V4 on-match next ! route-map FROM_BGP_PEER_V4 permit 11 match community allow_list_default_community ! route-map FROM_BGP_PEER_V4 permit 100 ! route-map FROM_BGP_PEER_V6 permit 10 call ALLOW_LIST_DEPLOYMENT_ID_0_V6 on-match next ! route-map FROM_BGP_PEER_V6 permit 11 match community allow_list_default_community ! route-map FROM_BGP_PEER_V6 permit 1 set ipv6 next-hop prefer-global on-match next ! route-map FROM_BGP_PEER_V6 permit 100 ! route-map TO_BGP_PEER_V4 permit 100 ! route-map TO_BGP_PEER_V6 permit 100 ! route-map RM_SET_SRC permit 10 ! ! agentx ! line vty ! $
以下は/etc/sonic/frr/staticd.confファイルの内容。
ファイル名のとおりスタティックルートの設定が保存されているファイルのようで、スタティックルートの設定以外は、zebra.confファイルおよびbgpd.confファイルと同じである。
write memoryコマンドで設定内容が反映される為、当ファイルは直接編集していない。
show runningconfig allコマンドの出力内容は、zebra.conf、bgpd.conf、staticd.confの各ファイルの内容をマージした内容と同じである事に気が付く。
$ sudo cat /etc/sonic/frr/staticd.conf ! ! Zebra configuration saved from vty ! 2021/03/20 02:00:24 ! frr version 7.5-sonic frr defaults traditional ! hostname jpmtkvmsw99 password zebra enable password zebra log syslog informational log facility local4 ! ! ! ip route 0.0.0.0/0 192.168.3.1 200 line vty ! $
BGPテーブルの内容の確認
ここに来てようやくBGPテーブルの内容を確認。
SONiC側のBGPテーブルの内容を確認。
設定内容のとおり、各物理リンクにてBGPピアを張っている事が分かる。
# show ip bgp summary IPv4 Unicast Summary: BGP router identifier 10.1.1.99, local AS number 65100 vrf-id 0 BGP table version 2 RIB entries 3, using 576 bytes of memory Peers 5, using 107 KiB of memory Peer groups 2, using 128 bytes of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt 10.0.0.1 4 65200 1209 1208 0 0 0 01:00:07 1 2 10.0.0.3 4 65200 1209 1208 0 0 0 01:00:07 1 2 10.0.0.5 4 65200 1208 1207 0 0 0 01:00:05 1 2 10.0.0.7 4 65200 1209 1208 0 0 0 01:00:07 1 2 10.0.0.9 4 65200 956 955 0 0 0 00:47:33 1 2 Total number of neighbors 5 #
Cumulus Linux側のBGPテーブルの内容を確認。
こちらも設定内容のとおり、各物理リンクにてBGPピアを張っている事が分かる。
$ net show bgp summary show bgp ipv4 unicast summary ============================= BGP router identifier 10.1.1.98, local AS number 65200 vrf-id 0 BGP table version 2 RIB entries 3, using 576 bytes of memory Peers 5, using 106 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd jpmtkvmsw99(10.0.0.0) 4 65100 1231 1231 0 0 0 01:01:16 1 jpmtkvmsw99(10.0.0.2) 4 65100 1231 1231 0 0 0 01:01:15 1 jpmtkvmsw99(10.0.0.4) 4 65100 1229 1230 0 0 0 01:01:14 1 jpmtkvmsw99(10.0.0.6) 4 65100 1231 1231 0 0 0 01:01:16 1 jpmtkvmsw99(10.0.0.8) 4 65100 978 979 0 0 0 00:48:42 1 Total number of neighbors 5 show bgp ipv6 unicast summary ============================= % No BGP neighbors found show bgp l2vpn evpn summary =========================== % No BGP neighbors found $
まとめ
Cumulus Linuxのでは、CiscoやJuniperのスイッチと同様にコマンドによる設定だけで目的を達成する事が出来た。
SONiCでは、開発版に関してはvtyshモードのコマンドによる設定と/etc/sonic/config_db.jsonファイルの編集で目的を達成する事が出来たが、製品版では/etc/sonic/config_db.jsonファイルを直接編集する必要は無いようだ。(下記リンク先を参照)
参照先など
開発者および先達の皆さまに感謝。
https://azure.github.io/SONiC/ SONiC
https://github.com/Azure/SONiC/wiki/Configuration SONiC Configuration
https://cumulusnetworks.com/ Cumulus Linux
https://docs.cumulusnetworks.com/knowledge-base/ Cumulus Networks Knowledge Base
https://debslink.hatenadiary.jp/entry/20210131/1612091391 SONiCを少しかじってみた
https://debslink.hatenadiary.jp/entry/20210214/1613279732 SONiCのWarm Bootの動作確認
https://debslink.hatenadiary.jp/entry/20240421/1713678733 DELL版のSONiCをVMware Fusionで動かす - 導入編
https://support.edge-core.com/hc/en-us/articles/900000789566-BGP-Step-1-Establish-BGP-Session 製品版SONiCのBGP設定例(Edgecore Networks社)
https://support.edge-core.com/hc/en-us/articles/900003896906-Enterprise-SONiC-Distribution-by-Edgecore 製品版SONiCのダウンロードページ(Edgecore Networks社)
