![[PukiWiki]](image/eucalyptus-j.png "[PukiWiki]")
Eucalyptus versions 1.5 and higher include a highly configurable VM networking subsystem that can be adapted to a variety of network environments. There are four high level networking "modes", each with its own set of configuration parameters, features, benefits and in some cases restrictions placed on your local network setup. The administrator must select one of these four modes before starting Eucalyptus on the front-end and nodes via modification of the 'eucalyptus.conf' configuration file on each machine running a Eucalyptus component. Brief descriptions of each mode follows:
Eucalyptus 1.5以降では、様々なネットワーク環境に適用でき高度にカスタマイズ可能な仮想ネットワーク機能が含まれています。Eucalyptusの仮想ネットワーク機能は大きく４つの「ネットワークモード」に分かれており、それぞれ固有の設定・機能・利点と制限を持っています。Eucalyptus管理者は４つのうち１つのネットワークモードを選択し適切に設定する必要があります。ネットワークモードの設定はフロントエンド・各ノードそれぞれの"eucalyptus.conf"設定ファイルに記述します。４つのネットワークモードについて、以下に簡単に内容を記述します。
Each Eucalyptus network mode has its own set of infrastructure requirements, configuration parameters, and caveats. These are described in more detail in the following sections.
Eucalyptus の各ネットワークモードには、それぞれで必要となるネットワークの条件や、個別の設定方法、注意事項があります。詳しくは、以下のセクションを御覧ください
For some of the network modes, you'll be required to set up an Ethernet bridge in order for the network mode to function. If you use Xen, the distros typically set up a bridge for you, and you'll simply have to find its name. For Xen versions 3.0 or earlier the bridge name is typically
いくつかのネットワークモードでは、イーサネットのブリッジを構成する必要があります。Xenを使っている場合、いくつかのディストリビューションでは自動的にブリッジインターフェイスが構成されており、やるべきことはそのデバイス名を特定することだけです。Xen 3.0以前では多くの場合は以下のようなデバイス名になっています。
xenbr0
while if you use Xen 3.2 the bridge name is typically
一方、Xen 3.2では以下のようなデバイス名になっています。
eth0
If you use KVM, or you wish to configure a bridge manually, the following describes how to set up a bridge on various distributions. In these examples, it is assumed that your bridge device will obtain its IP address using DHCP, and that your physical ethernet device is named 'eth0'.
KVMを使っていてブリッジネットワークを手動で構成する場合は、いくつかのディストリビューションでの設定方法を以下に記述しています。以下の例では、物理インターフェイスのデバイス名は'eth0'で、ブリッジインターフェイスはDHCPでIPアドレスを取得することを想定しています。
# create a new ethernet bridge configuration file '/etc/sysconfig/network-scripts/ifcfg-br0' # and populate it with the following # この記述を/etc/sysconfig/network-scripts.ifcfg-br0として記述します DEVICE=br0 BOOTPROTO=dhcp ONBOOT=yes TYPE=Bridge # add your physical ethernet device to the bridge by editing your physical ethernet device # configuration file (in this example, 'eth0') '/etc/sysconfig/network-scripts/ifcfg-eth0' # 以下のような記述を/etc/sysconfig/network-scripts/ifcfg-eth0に追加します DEVICE=eth0 TYPE=Ethernet BRIDGE=br0
# All network configuration is done using the 'yast2' configuration tool Run yast2 Network Devices Network Settings Add Device Type->Bridge Next Bridged Devices->select your pysical device (eth0) Next Ok Quit
# 'yast2'設定ツールで以下のように操作します yast2 を実行する Network Devices を選択 Network Settings を選択 追加 を選択 Device Type に Bridge を選択 次へ を選択 Bridged Devices に select your physical device (eth0) を選択 次へ を選択 Ok を選択 終了 を選択
# create a new ethernet bridge device and attach your physical
# ethernet device to the bridge by adding the following to '/etc/network/interfaces'
# 以下のような記述を/etc/network/interfacesに追加し、
# 新規にブリッジネットワークデバイスを作成し、物理ネットワークインターフェイスに接続します
auto br0
iface br0 inet dhcp
bridge_hello 2
bridge_fd 1
bridge_ports eth0
# and comment out your settings for 'eth0' in the same file
# 同じファイル内の'eth0'に関わる設定項目はコメントアウトします
When you are finished configuring a bridge, it is advised that you restart the machine (or, at least, restart the networking subsystem). Once it is back up and running, the
上記の設定が終わったら、マシンを再起動します(もしくは少なくともネットワークの再起動を行います)。再起動後、以下のようなコマンドを実行します。
brctl show
command will list all available bridges, which you can use to check that your system is properly configured to run Eucalyptus.
brctlコマンドは使用可能なブリッジインターフェイスの一覧を表示します。上記で設定したブリッジインターフェイスが表示されているかどうかを確認してください。
NOTE: the bridge name
virbr0
is created by libvirt is shouldn't not be used.
For the reminder of this document, we assume that you correctly identified the bridge and that such bridge is called
br0
注意：ブリッジインターフェイス 'virbr0' はlibvirtにより作成されたブリッジインターフェイスですが、使用できません。この手順では正しいブリッジインターフェイスは 'br0'であることを前提に記述しています。
There is very little Eucalyptus configuration to use SYSTEM mode, as in this mode, Eucalyptus mostly stays 'out of the way' in terms of VM networking. The options in 'eucalyptus.conf' that must be configured correctly in 'SYSTEM' mode are as follows:
EucalyptusクラウドをSYSTEMモードで動作させる場合の設定は非常にシンプルです。SYSTEMモードで動作する場合、インスタンスのネットワークについてEucalyptusはほとんど関与しません。EucalyptusをSYSTEMモードで動作させる場合、eucalyptus.confに以下のように記述します。
On the front-end:
フロントエンドでの記述内容
VNET_MODE="SYSTEM"
On each node:
ノードでの記述内容
VNET_MODE="SYSTEM" VNET_BRIDGE
In each Eucalyptus node controller's (NC) 'eucalyptus.conf' file, make sure that the parameter 'VNET_BRIDGE' is set to the name of the bridge device that is connected to your local ethernet
各ノード(ノード・コントローラが動作するマシン)のeucalyptus.confの 'VNET_BRIDGE' には正しく物理インターフェイスに接続されたブリッジインターフェイスを指定します。以下の例ではブリッジインターフェイス 'br0' を使用しています。
VNET_BRIDGE="br0"
Make sure that what you are specifying in this field is actually a bridge, and that it is the bridge that is connected to an ethernet network that has a DHCP server running elsewhere that is configured to hand out IP addresses dynamically. Note that your front-end machine does not need to have any bridges (this is fine, as VNET_BRIDGE is only a relevant for node controllers, and will be safely ignored by the front-end components).
この場所で指定しているものが、実際のブリッジに当たります。ブリッジ(と、それが接続されている物理ネットワークインターフェイス)は、同一LAN内に配置されているDHCP サーバと更新できる必要があります。
フロントエンドではブリッジを設定する必要がありません。VNET_BRIDGE はノード・コントローラに対してのみ設定可能な項目であり、フロントエンドでは無視されます。
To test whether this mode is working properly at run-time, you can check on a node before and after an instance is running the configure bridge. You should see a new interface associate with the bridge for example you could see
正常に SYSTEMモードが動作するかを確認するためには、brctl showコマンドを使用します。brctl showコマンドはEucalyptusを動作させる前でも、後でも実行することができます。(訳注：が、無用のトラブルを避けるためにも、Eucalyptus実行前に確認しておく方がよいでしょう)
; brctl show
bridge name bridge id STP enabled interfaces
eth0 8000.000c29369858 no peth0
vif18.0
on a node controller running Xen 3.2: note that Eucalyptus has correctly attached the VM's 'eth0' interface (vif18.0) to the bridge ('br0') that is being used to attach VMs to the local ethernet ('peth0').
ノードがXen3.2で動作している場合、インスタンスのネットワークインターフェイス'eth0(vif18.0)'の接続先は'br0'ですが、上記例では物理ネットワークインターフェイスである'peth0'に接続されていることになります。
In the case of kvm you may see something like
KVMを使用している場合は出力は以下のようになります。
; brctl show bridge name bridge id STP enabled interfaces br0 8000.00005a00083d no eth0 vnet0
At this point, the VM should be sending DHCP requests to the local ethernet, and the DHCP server on the network should be sending a reply.
この時点で、インスタンスはDHCPリクエストを発行し、DHCPサーバの応答を受け取ることができる状態になっているはずです。
CAVEATS - In this mode, as mentioned previously, VMs are simply started with their ethernet interfaces attached to the local ethernet without any isolation. Practically, this means that you should treat a VM the same way that you would treat a non-VM machine running on the network. Eucalyptus does it's best to discover the IP address that was assigned to a running VM via a third-party DHCP server, but can be unsuccessful depending on the specifics of your network (switch types/configuration, location of CC on the network, etc.). Practically, if Eucalyptus cannot determine the VM's IP, then the user will see '0.0.0.0' in the output of 'describe-instances' in both the private and public address fields. The best workaround for this condition is to instrument your VMs to send some network traffic to your front end on boot (after they obtain an IP address). For instance, setting up your VM to ping the front-end a few times on boot should allow Eucalyptus to be able to discover the VMs IP.
警告 - SYSTEM モードでは、上記のように、仮想マシン間のネットワークは隔離されていません。インスタンスのネットワークの挙動は、インスタンスではない物理的なマシンと全く等価です。インスタンスは、DHCP サーバからのをIP アドレス割り当てを受けるよう努力しますが、ネットワークの設定によっては見つけられないこともあります（スイッチの種類や設定、ネットワーク上のクラウド・コントローラの位置などによります）。実際のところ、Eucalyptus が仮想マシンの IP アドレスを特定できない場合、'describe-instanse' コマンドによって表示される IP アドレスは「0.0.0.0」（private 及び public の両方とも）となります。このような状況を避ける一番良い方法は、仮想マシンの起動時（正確には、起動後の IP アドレス取得時）に、フロントエンドに対して若干のネットワーク通信を発生させることです。Eucalyptus が仮想マシンの IP アドレスを発見できるようにするためには、仮想マシンの起動時、仮想マシンからフロントエンド側に数回の ping を飛ばしてみてください。
In this mode, Eucalyptus will manage VM IP address assignment by maintaining its own DHCP server with one static entry per VM. The options in 'eucalyptus.conf' that must be configured correctly in 'STATIC' mode are as follows:
On the front-end (options annotated with a '*' may be required depending on your installation, see below for details):
VNET_MODE="STATIC" VNET_PUBINTERFACE VNET_PRIVINTERFACE VNET_DHCPDAEMON *VNET_DHCPUSER [#ic567933] VNET_SUBNET VNET_NETMASK VNET_BROADCAST VNET_ROUTER VNET_DNS VNET_MACMAP
On each node:
VNET_MODE="STATIC" VNET_BRIDGE
The Eucalyptus administrator must configure the front-end's 'eucalyptus.conf' first with a valid, configured ethernet device that is attached to the same physical ethernet as the Eucalyptus nodes:
VNET_PRIVINTERFACE="eth0"
If the front-end has a second ethernet device which is used to access the public network, let's say eth1, then you need to configured it as
VNET_PUBINTERFACE="eth1"
Next, the admin must ensure that there is a DHCP server binary installed on the front-end and Eucalyptus knows where it is located:
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
If your DHCP daemon binary is configured to run as 'non-root' (say, as the user 'dhcpd' as is the case in Ubuntu >= 8.10), then you must configure Eucalyptus to be aware of that user:
VNET_DHCPUSER="<dhcpusername>"
Then, the admin must input IP subnet information for that device. For example, if the front-end's 'eth0' interface has the IP address '192.168.1.254' on the '192.168.1.0/24' network, with a gateway at '192.168.1.1' and a DNS at '192.168.1.2', the values in 'eucalyptus.conf' would look like so:
VNET_SUBNET="192.168.1.0" VNET_NETMASK="255.255.255.0" VNET_BROADCAST="192.168.1.255" VNET_ROUTER="192.168.1.1" VNET_DNS="192.168.1.2"
Finally, the administrator must supply a list of static MAC/IP mappings that will be assigned, first come first served, to VM instances. Note that each IP must reside in the subnet defined above, and must not be in use by any other machine on the network.
VNET_MACMAP="AA:DD:11:CE:FF:ED=192.168.1.3 AA:DD:CE:FF:EE=192.168.1.4"
On the nodes, you must ensure that the bridge is entered
VNET_BRIDGE="br0"
Once you have configured Eucalyptus properly, start up the node controllers and the front-end components. To test whether this mode is working properly at run-time, you can follow the last paragraph of the SYSTEM mode, in which the bridge is inspected.
Make sure that the DHCP server has been started properly on the front-end ('ps axww | grep -i dhcpd | grep -i euca'). At this point, the VM should be sending DHCP requests to the local ethernet, and the DHCP server on the front-end should be sending a reply with one of the static MAC/IP mappings the admin has defined in 'eucalyptus.conf'.
CAVEATS - In this mode, as mentioned previously, VMs are started with their ethernet interfaces attached to the local ethernet without any isolation. Practically, this means that you should treat a VM the same way that you would treat a non-VM machine running on the network. Eucalyptus does not verify that your settings are valid, thus, you must enter them correctly in order for your VMs to obtain IP addresses. Finally, we assume that the installed DHCP daemon is, or is compatible with, ISC DHCP Daemon version 3.0.X. If it is not, we recommend either installing a version that is (common in most distributions) or writing a wrapper script around your installed DHCP server and point Eucalyptus at it (via VNET_DHCPDAEMON in 'eucalyptus.conf').
In this mode, Eucalyptus will fully manage the local VM instance network and provides all of the networking features Eucalyptus current supports (VM network isolation, user controllable VM firewalls (ingress rules/security groups), dynamic public IP assignment). The options in 'eucalyptus.conf' that must be configured correctly in 'MANAGED' mode are as follows:
On the front-end (options annotated with a '*' may be required depending on your installation, see below for details):
VNET_MODE="MANAGED" VNET_PUBINTERFACE VNET_PRIVINTERFACE VNET_DHCPDAEMON *VNET_DHCPUSER [#hc00988f] VNET_SUBNET VNET_NETMASK VNET_DNS VNET_ADDRSPERNET *VNET_PUBLICIPS [#z1d27057] *VNET_CLOUDIP [#mad9196c] *VNET_LOCALIP [#e767369d]
On each node:
VNET_MODE="MANAGED" VNET_PUBINTERFACE VNET_PRIVINTERFACE
Be advised that this mode requires that your local network/configuration conforms to certain requirements that Eucalyptus depends upon.
Before using 'MANAGED' mode, you must confirm that:
All three of these requirements must be met before MANAGED mode should be attempted. Failure to verify the above will, at least, result VM instances being unavailable on the network.
For requirement '1', choose a IP range that you know is completely unused on your network. Choose a range that is as large as possible. Typical examples are:
if the network 10.0.0.0 - 10.255.255.255 is completely unused:
VNET_MODE="MANAGED" VNET_SUBNET="10.0.0.0" VNET_NETMASK="255.0.0.0" VNET_DNS="<your DNS>" VNET_ADDRSPERNET="128"
or if the network 192.168.0.0 - 192.168.255.255 is completely unused:
VNET_MODE="MANAGED" VNET_SUBNET="192.168.0.0" VNET_NETMASK="255.255.0.0" VNET_DNS="<your DNS>" VNET_ADDRSPERNET="64"
Next, the admin must verify that the local network will allow/forward VLAN tagged packets between machines running Eucalyptus components. To verify, perform the following test:
on the front-end, choose the interface that is on the local ethernet (and will be set in eucalyptus.conf as VNET_PRIVINTERFACE), and run:
vconfig add <interface> 10 ifconfig <interface>.10 192.168.1.1 up
replace '192.168.1.1' with an IP from the range you selected above.
On the node, choose the interface on the local network (will be set in eucalyptus.conf as VNET_PRIVINTERFACE and VNET_PUBINTERFACE), and run:
vconfig add <interface> 10 ifconfig <interface>.10 192.168.1.2 up
again, replace '192.168.1.2' with another IP in the range you selected above.
Then, try a ping between hosts. On the front-end:
ping 192.168.1.2
on the node:
ping 192.168.1.1
If this does not work, then your switch needs to be configured to forward VLAN tagged packets (if it is a managed switch, see your switch's documentation to determine how to do this).
Finally, you need to carefully inspect the firewall on the front-end to make sure that it will not interfere with Eucalyptus, or vice-versa. Eucalyptus will flush the 'filter' and 'nat' tables upon boot in MANAGED mode, but provides a way for the administrator to define special rules that are loaded when Eucalyptus starts (see below for details).
The Eucalyptus administrator must configure the front-end's 'eucalyptus.conf' first with a valid, configured ethernet device that is attached to the same physical ethernet as the Eucalyptus nodes:
VNET_PRIVINTERFACE="eth0"
If the front-end has a second ethernet device which is used to access the public network, let's say eth1, then you need to configured it as
VNET_PUBINTERFACE="eth1"
Next, the admin ust ensure that there is a DHCP server binary installed on the front-end and Eucalyptus knows where it is located:
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
If your DHCP daemon binary is configured to run as 'non-root' (say, as the user 'dhcpd' as is the case in Ubuntu >= 8.10), then you must configure Eucalyptus to be aware of that user:
VNET_DHCPUSER="<dhcpusername>"
Nodes must have VNET_PRIVINTERFACE and VNET_PUBINTERFACE set properly (they should be set with the same value, the device facing the CC). For example, with current Xen versions, this parameter (when your node's Xen bridge is 'eth0') is typically:
VNET_PUBINTERFACE="peth0" VNET_PRIVINTERFACE="peth0"
while for kvm it should be something like
VNET_PUBINTERFACE="eth0" VNET_PRIVINTERFACE="eth0"
Once you have verified that your network configuration meets the requirements for running in MANAGED mode, the rest of the configuration is fairly simple. For example, if the 192.168.0.0/16 network is free and unused on your network:
VNET_MODE="MANAGED" VNET_SUBNET="192.168.0.0" VNET_NETMASK="255.255.0.0" VNET_DNS="<your dns>" VNET_ADDRSPERNET="64" VNET_PUBLICIPS="<publicIPa> <publicIPb> ... <publicIPn>"
SUBNET, NETMASK, and DNS have been described previously. VNET_ADDRSPERNET is used to control how many VM instances may simultaneously be part of an individual user's named network (called a 'security group' in Amazon EC2). Choosing the right value for this parameter depends on how many IPs you have made available using VNET_SUBNET/VNET_NETMASK, how many VLANs your network supports simultaneously, and how many concurrent active user networks the administrator wishes to support. In the above example, there are 65536 addresses available (192.168.0.0/16). If we divide by the number of addresses per network (set to 64 above), we find the maximum number of simultaneous active named networks that can be in use at any one point in time (65536 / 64 == 1024). If your eucalyptus installation has 100 users, then each user could have at most 10 active security groups in operation at any point in time (of course, they can define as many as they wish, but can only have sets of running VMs residing in at most 10 networks). Each security group could support up to 61 instances (64 addresses minus 1 address for the subnet, broadcast, and router IPs). If your installation favors more VMs per network and fewer active security groups per user, the administrator may adjust the VNET_ADDRSPERNET parameter accordingly. Setting it to '256' would result in each active user's security group supporting up to 253 VM instances, and each of 100 users could simultaneously have 2 active security groups.
If you would like users to log in to their instances from outside the cluster/cluster front-end, you must find a set of public IP addresses, that are not in use, and allow Eucalyptus to dynamically route them to VM instances at instance boot time or dynamically at run time. For each IP address you choose, your front-end must be capable of being configured with that IP address. To test, choose some free public IP addresses and perform the following test for each one:
on the front-end:
ip addr add <publicIP>/32 dev <interface>
on some external machine representative of where users will wish to log into their VM instances:
ping <publicIP>
if this works, then dynamic IP assignment to VM instances will work. Remove the assigned address with the following command:
ip addr del <publicIP>/32 dev <interface>
Once you have compiled a list of available public IP addresses, allow Eucalyptus to use them by listing the IPs in 'eucalyptus.conf':
VNET_PUBLICIPS="<publicIPa> <publicIPb> ... <publicIPn>"
or, you can specify a range of IPs with:
VNET_PUBLICIPS="<publicIPa>-<publicIPb>"
where publicIPa and publicIPb are in the same /24 subnet.
If your cluster-controller and cloud-controller are running on separate hosts, you need to set:
VNET_CLOUDIP="<ip-of-cloud-controller>"
And, if you are running multiple clusters in your installation, and wish to manually specify the IP of the cluster-controller that all other cluster-controllers can reach, you may set:
VNET_LOCALIP="<ip-of-cluster-controller"
The cluster-controller will attempt to determine this value automatically if it is not set.
CAVEATS - When Eucalyptus is running in MANAGED mode, you cannot currently run an entire eucalyptus installation on a single machine as this mode depends upon traffic between named networks passing through a front-end router (instead of going through the loopback device). If you wish to run Eucalyptus on a single machine (laptop), you must use SYSTEM or STATIC mode. In MANAGED mode, Eucalyptus will flush the front-end's iptables rules for both 'filter' and 'nat'. Next, it will set the default policy for the 'FORWARD' chain in 'filter' to 'DROP'. At run time, the front-end will be adding and removing rules from 'FORWARD' as users add/remove ingress rules from their active security groups. In addition, the 'nat' table will be configured to allow VMs access to the external network using IP masquerading, and will dynamically add/remove rules in the 'nat' table as users assign/unassign public IPs to VMs at instance boot or run-time. If the administrator has some rules that they wish to apply o the front-end, they should perform the following procedure on the front-end, before eucalyptus is started or while eucalyptus is not running. WARNING if the admin chooses to perform this operation to define special iptables rules that are loaded when Eucalyptus starts, they could inadvertently cause Eucalyptus VM networking to fail. It is suggested that you only do this only if you are completely sure that it will not interfere with the operation of Eucalyptus.
<use iptables to set up your iptables rules> iptables-save > $EUCALYPTUS/var/run/eucalyptus/net/iptables-preload
If you start an instance believe that it is running but is not available on the network, here are some things to check.
First, verify that the requirements of MANAGED mode have been met as described above (unused range of IPs, VLAN capable network, no interfering firewall rules on the nodes or front-end). Test whether you can get to the instance from the front-end using it's private address (from the range you specified). If you cannot, next, inspect the interfaces on the front-end and nodes:
on front-end:
ifconfig -a
You should see an interface '<interface>.<vlan>' with an IP address that is up and running. For instance, if may be 'eth0.10'. If it is not, check your VNET_PUBINTERFACE and VNET_PRIVINTERFACE parameter and inspect the eucalyptus log files for errors.
on the node:
brctl show
You should see a number of bridges called 'eucabr<vlan>', where '<vlan>' is a number that typically starts from '10'. The output should be similar (if VNET_PRIVINTERFACE="peth0") to:
; brctl show
bridge name bridge id STP enabled interfaces
eucabr10 8000.000c29369858 no peth0.10
vif18.0
If this is not the case, check your VNET_PRIVINTERFACE setting, and inspect the logfiles for details.
Back on the front-end, make sure that 'dhcpd' is running:
ps axww | grep <dhcpd>
where '<dhcpd>' is what you have set for VNET_DHCPDAEMON. Make sure that, in the output of 'ps', you see that the daemon is listening on the vlan tagged interface from above (<interface>.<vlan>). If it is not running, check the eucalyptus logs for the reason why (if the command failed, you will see this information in 'cc.log', if the daemon failed at runtime, you can inspect the reason in the daemon's output itself in 'http-cc_error_log'.
If you can access the private IP of the instance from the front-end, but public IPs are not being forwarded properly, first confirm that the user's security group is set up properly by having them run 'euca-describe-group <group of instance>'. '<group of instance>' is set to 'default' by default or if unspecified when the instance was started. If the group has appropriate ingress rules set, check that the rules have been implemented on the front-end:
iptables -L <username>-<groupname>
If there are no rules here, check the 'cc.log' for errors applying the table rules for more insight. Next, check the 'nat' table:
iptables -L -t nat
You should see one DNAT rule for routing traffic from a public IP to the instance IP, and one SNAT rule for setting the source IP of outgoing packets from that instance. If you do not, check 'cc.log' to determine the cause.
If all of these checks pass and the instance still is experiencing network problems, please prepare the following information and send it along to the Eucalyptus discussion board:
on front-end and one representative node, capture the output of the following commands:
netstat -rn ifconfig -a brctl show iptables-save
and send us 'cc.log', 'nc.log', 'httpd-cc_error_log' and 'httpd-nc_error_log'.
In this mode, Eucalyptus will fully manage the local VM instance network and provides all of the networking features Eucalyptus current supports (user controllable VM firewalls (ingress rules/security groups), dynamic public IP assignment), but does not provide VM network isolation. The options in 'eucalyptus.conf' that must be configured correctly in 'MANAGED-NOVLAN' mode are as follows:
On the front-end (options annotated with a '*' may be required depending on your installation, see below for details):
VNET_MODE="MANAGED-NOVLAN" VNET_PRIVINTERFACE VNET_PUBINTERFACE VNET_DHCPDAEMON *VNET_DHCPUSER [#d7ee0a79] VNET_SUBNET VNET_NETMASK VNET_DNS VNET_ADDRSPERNET *VNET_PUBLICIPS [#b78249a3] *VNET_CLOUDIP [#s6f77bde] *VNET_LOCALIP [#ge23a724]
On each node:
VNET_MODE="MANAGED-NOVLAN" VNET_BRIDGE
Be advised that this mode requires that your local network/configuration conforms to certain requirements that Eucalyptus depends upon.
Before using 'MANAGED-NOVLAN' mode, you must confirm that:
Both of these requirements must be met before MANAGED-NOVLAN mode should be attempted. Failure to verify the above will, at least, result VM instances being unavailable on the network.
For requirement '1', choose a IP range that you know is completely unused on your network. Choose a range that is as large as possible. Typical examples are:
if the network 10.0.0.0 - 10.255.255.255 is completely unused:
VNET_MODE="MANAGED-NOVLAN" VNET_SUBNET="10.0.0.0" VNET_NETMASK="255.0.0.0" VNET_DNS="<your DNS>" VNET_ADDRSPERNET="128"
or if the network 192.168.0.0 - 192.168.255.255 is completely unused:
VNET_MODE="MANAGED-NOVLAN" VNET_SUBNET="192.168.0.0" VNET_NETMASK="255.255.0.0" VNET_DNS="<your DNS>" VNET_ADDRSPERNET="64"
You will need to carefully inspect the firewall on the front-end to make sure that it will not interfere with Eucalyptus, or vice-versa. Eucalyptus will flush the 'filter' and 'nat' tables upon boot in MANAGED-NOVLAN mode, but provides a way for the administrator to define special rules that are loaded when Eucalyptus starts (see below for details).
The Eucalyptus administrator must configure the front-end's 'eucalyptus.conf' first with a valid, configured ethernet device that is attached to the same physical ethernet as the Eucalyptus nodes:
VNET_PRIVINTERFACE="eth0"
If the front-end has a second ethernet device which is used to access the public network, let's say eth1, then you need to configured it as
VNET_PUBINTERFACE="eth1"
Next, the admin ust ensure that there is a DHCP server binary installed on the front-end and Eucalyptus knows where it is located:
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
If your DHCP daemon binary is configured to run as 'non-root' (say, as the user 'dhcpd' as is the case in Ubuntu >= 8.10), then you must configure Eucalyptus to be aware of that user:
VNET_DHCPUSER="<dhcpusername>"
Nodes must have VNET_BRIDGE set properly:
VNET_BRIDGE="br0"
Once you have verified that your network configuration meets the requirements for running in MANAGED-NOVLAN mode, the rest of the configuration is fairly simple. For example, if the 192.168.0.0/16 network is free and unused on your network:
VNET_MODE="MANAGED-NOVLAN" VNET_SUBNET="192.168.0.0" VNET_NETMASK="255.255.0.0" VNET_DNS="<your dns>" VNET_ADDRSPERNET="64" VNET_PUBLICIPS="<publicIPa> <publicIPb> ... <publicIPn>"
SUBNET, NETMASK, and DNS have been described previously. VNET_ADDRSPERNET is used to control how many VM instances may simultaneously be part of an individual user's named network (called a 'security group' in Amazon EC2). Choosing the right value for this parameter depends on how many IPs you have made available using VNET_SUBNET/VNET_NETMASK, how many VLANs your network supports simultaneously, and how many concurrent active user networks the administrator wishes to support. In the above example, there are 65536 addresses available (192.168.0.0/16). If we divide by the number of addresses per network (set to 64 above), we find the maximum number of simultaneous active named networks that can be in use at any one point in time (65536 / 64 == 1024). If your eucalyptus installation has 100 users, then each user could have at most 10 active security groups in operation at any point in time (of course, they can define as many as they wish, but can only have sets of running VMs residing in at most 10 networks). Each security group could support up to 61 instances (64 addresses minus 1 address for the subnet, broadcast, and router IPs). If your installation favors more VMs per network and fewer active security groups per user, the administrator may adjust the VNET_ADDRSPERNET parameter accordingly. Setting it to '256' would result in each active user's security group supporting up to 253 VM instances, and each of 100 users could simultaneously have 2 active security groups.
If you would like users to log in to their instances from outside the cluster/cluster front-end, you must find a set of public IP addresses, that are not in use, and allow Eucalyptus to dynamically route them to VM instances at instance boot time or dynamically at run time. For each IP address you choose, your front-end must be capable of being configured with that IP address. To test, choose some free public IP addresses and perform the following test for each one:
on the front-end:
ip addr add <publicIP>/32 dev <interface>
on some external machine representative of where users will wish to log into their VM instances:
ping <publicIP>
if this works, then dynamic IP assignment to VM instances will work. Remove the assigned address with the following command:
ip addr del <publicIP>/32 dev <interface>
Once you have compiled a list of available public IP addresses, allow Eucalyptus to use them by listing the IPs in 'eucalyptus.conf':
VNET_PUBLICIPS="<publicIPa> <publicIPb> ... <publicIPn>"
or, you can specify a range of IPs with:
VNET_PUBLICIPS="<publicIPa>-<publicIPb>"
where publicIPa and publicIPb are in the same /24 subnet.
If your cluster-controller and cloud-controller are running on separate hosts, you need to set:
VNET_CLOUDIP="<ip-of-cloud-controller>"
And, if you are running multiple clusters in your installation, and wish to manually specify the IP of the cluster-controller that all other cluster-controllers can reach, you may set:
VNET_LOCALIP="<ip-of-cluster-controller"
The cluster-controller will attempt to determine this value automatically if it is not set.
CAVEATS - When Eucalyptus is running in MANAGED-NOVLAN mode, you cannot currently run an entire eucalyptus installation on a single machine as this mode depends upon traffic between named networks passing through a front-end router (instead of going through the loopback device). If you wish to run Eucalyptus on a single machine (laptop), you must use SYSTEM or STATIC mode. In MANAGED-NOVLAN mode, Eucalyptus will flush the front-end's iptables rules for both 'filter' and 'nat'. Next, it will set the default policy for the 'FORWARD' chain in 'filter' to 'DROP'.
At run time, the front-end will be adding and removing rules from 'FORWARD' as users add/remove ingress rules from their active security groups. In addition, the 'nat' table will be configured to allow VMs access to the external network using IP masquerading, and will dynamically add/remove rules in the 'nat' table as users assign/unassign public IPs to VMs at instance boot or run-time. If the administrator has some rules that they wish to apply o the front-end, they should perform the following procedure on the front-end, before eucalyptus is started or while eucalyptus is not running. WARNING if the admin chooses to perform this operation to define special iptables rules that are loaded when Eucalyptus starts, they could inadvertently cause Eucalyptus VM networking to fail. It is suggested that you only do this only if you are completely sure that it will not interfere with the operation of Eucalyptus.
<use iptables to set up your iptables rules> iptables-save > $EUCALYPTUS/var/run/eucalyptus/net/iptables-preload
If you edit a networking related value in eucalyptus.conf, you will need to restart the CC ($EUCALYPTUS/etc/init.d/eucalyptus-cc restart) for changes to take effect. If you change the networking mode, you will need to perform a cleanrestart ($EUCALYPTUS/etc/init.d/eucalyptus-cc cleanrestart)
If you are running Eucalyptus in multi-cluster mode, we strongly recommend that you configure all your clusters to have an identical networking mode. In addition we strongly recommend that you do not run multiple clusters in the same broadcast domain. Each cluster should be in a separate domain.
Eucalyptus versions >= 1.6 support multiple clusters within a single Eucalyptus cloud installation. This section briefly describes how Eucalyptus manages the networking aspect of a multi-cluster setup. First, in SYSTEM or STATIC networking modes, Eucalyptus does not perform any special configuration for a multi-cluster setup. In MANAGED and MANAGED-NOVLAN modes, Eucalyptus will set up layer two tunnels between your clusters, so that virtual machines that are in the same security group, but distributed across clusters (potentially each in their own broadcast domain), can communicate with one another. We use the 'vtun' package to handle all layer two tunneling between clusters.
For the most part, as long as 'vtun' is installed on your cluster controllers, multi-cluster tunneling is handled automatically by the cluster controller software. There are a few caveats to be aware of, depending on your chosen networking mode and network topology.
MANAGED mode: during normal operation, you will see many tunnel interfaces being created and destroyed as virtual networks are constructed and torn down.
MANAGED-NOVLAN mode: your CC will need to be configured with a bridge as it's primary, public interface (VNET_PUBINTERFACE) in order for vtun tunneling to work in this mode.
BOTH modes: the CC attempts to auto-discover it's list of local IP addresses upon startup, but if the IP that was used to register the CC is not locally available, you can override the CC's notion of 'self' by setting the 'VNET_LOCALIP' variable in eucalyptus.conf.
BOTH modes: do not run two CCs in the same broadcast domain with tunneling enabled, this will potentially lead to a broadcast storm as tunnels start forwarding packets in a loop on your local network.
If you wish to disable tunneling altogether, set 'VNET_LOCALIP=0.0.0.0' in eucalyptus.conf.
« Backup of Eucalyptus (1.6)UPManagement »
Printer-friendly version Login or register to post co
