Cabling and Networking¶

The first step is always to cable and setup the networking as you can only then define how you configure Occam. We recommend running at least 5 servers for your OpenStack cloud. (Note: If you choose to use high-availability feature, you will need to add 2 more servers.)

With 5 servers you will have the following breakdown of your nodes:

• Ops Node
• Controller Node
• 2 Compute nodes
• Monitor Node

You will need to configure four networks.

Now we will go into how you need to cable these servers and also how to setup the necessary networking on both the router and the switch.

Cable Your Servers¶

Follow these steps to cable your servers:

1. Connect the iLO interfaces of the servers to one set of ports on the switch (usually dedicated ports, see your own documentation)
2. Select and take note of a set of ports on the switch that you would like to handle the Management Network. Connect all of these ports to the eth0 interface on the servers.
3. Decide which nodes will act as your controller, ops and monitor node. You will need this information later.
4. Connect the eth1 interface on the controller node to the switch. Take note of the port on the switch. It will serve the Public Network.
5. Connect one twinax interface of each server to the switch and make note of the ports on the switch. Note: the specific interface on the server will vary on the vendor. For our servers, it was eth5 but you will need to confirm which specific interface it will be on your servers.
6. Connect one interface from the router to the switch. Make note of the port you use on both the router and the switch.

Configure The Router And Switch¶

The router will have the gateways for the Management and Public Network configured along with VLAN tags created. Make sure that these networks can reach out to the internet using NATing.

Below you will find examples for how to configure the router and switch. In these examples we are assuming that the router is Juniper MX-240 and the switch is Arista 7050.

The examples also assume these assignments for the various networks:

It is also assumed that the interface on router which connected to switch is ge-0/2/0.

user@host# edit
user@host# set interfaces ge-0/2/0 unit 100 description "Link for Management Network"
user@host# set interfaces ge-0/2/0 unit 100 vlan-id 100
user@host# set interfaces ge-0/2/0 unit 100 family inet address 10.100.1.1/24
user@host# set interfaces ge-0/2/0 unit 200 description "Link for Public Network"
user@host# set interfaces ge-0/2/0 unit 200 vlan-id 200
user@host# set interfaces ge-0/2/0 unit 200 family inet address 10.101.2.1/24

user@host# show interfaces ge-0/2/0

ge-0/2/0 {
  unit 100 {
    description "Link for Management Network";
    vlan-id 100;
    family inet {
      address 10.100.1.1/24;
    }
  }
  unit 200 {
    description "Link for Public Network";
    vlan-id 200;
    family inet {
      address 10.101.2.1/24;
    }
  }
}

user@host# edit
user@host# set services nat pool NAT-POOL address <your public IP>/32
user@host# set services nat pool NAT-POOL port automatic

user@host# edit services nat pool NAT-POOL
user@host# show

address <your public IP>/32;
port {
  automatic;
}

user@host# set services nat rule NAT-RULE match-direction input
user@host# set services nat rule NAT-RULE term NAT from source-address 10.100.1.0/24
user@host# set services nat rule NAT-RULE term NAT from source-address 10.101.2.0/24
user@host# set services nat rule NAT-RULE term NAT then translated source-pool NAT-POOL
user@host# set services nat rule NAT-RULE term NAT then translated translation-type napt-44
user@host# set services nat rule NAT-RULE term NAT then translated mapping-type endpoint-independent
user@host# set services nat rule NAT-RULE term NAT then translated filtering-type endpoint-independent

user@host# set services nat rule NAT-RULE term Private from source-address 10.100.1.0/24
user@host# set services nat rule NAT-RULE term Private from source-address 10.101.2.0/24
user@host# set services nat rule NAT-RULE term Private from destination-address 10.100.1.0/24
user@host# set services nat rule NAT-RULE term Private from destination-address 10.101.2.0/24
user@host# set services nat rule NAT-RULE term Private then no-translation

user@host# edit services nat rule NAT-RULE
user@host# show

match-direction input;

term NAT {
  from {
    source-address {
      10.100.1.0/24;
      10.101.2.0/24;
    }
  }
  then {
    translated {
      source-pool NAT-POOL;
        translation-type {
          napt-44;
        }
        mapping-type endpoint-independent;
        filtering-type {
          endpoint-independent;
        }
      }
    }
  }
}

term Private {
  from {
    source-address {
      10.100.1.0/24;
      10.101.2.0/24;
    }
  }
  then {
    translated {
      source-pool NAT-POOL;
        translation-type {
        napt-44;
      }
      mapping-type endpoint-independent;
      filtering-type {
        endpoint-independent;
      }
    }
  }
}

Configure the Switch¶

(Fix this section) Create the vlan tags for iLO and VM Traffic Network

user@host (config)# vlan 300
user@host(config-vlan-300)# exit
user@host (config)# vlan 400
user@host(config-vlan-400)# exit

The switch will have virtual interfaces for the gateways of the VM and iLO networks. Apply the corresponding vlan tags to the iLO interfaces and VM Traffic interfaces. These will be access ports. See below for example of access port configuration.

Since the vlan tags for Management and Public Networks were created on the router, take note of them and assign the same vlan tags on the switch. The port on the switch that is connected to the router should be configured as a trunk port. In the example below we used 300 and 301 respectively. Assuming interface 48 on the switch as the port connected to the router, run the following commands:

user@host (config)# int eth48
user@host (config-if-Et48)# switchport mode trunk
user@host (config-if-Et48)# switch port trunk allowed vlan add 100,200
user@host (config-if-Et48)# exit

On the ports connected to iLO, simply configure the switch as access ports and assign a vlan tag. This vlan tag cannot be the same as the vlan tags used for the port trunk. Assuming two ports for iLO (1,2) and two ports VM Traffic (10,11):

user@host (config)# int eth 1-2
user@host (config-if-Et1-2)# description "Configuration for iLO, VLAN 400, 10.0.2.0/24"
user@host (config-if-Et1-2)# switchport access vlan 400
user@host (config-if-Et1-2)# exit

user@host (config)# int eth 10-11
user@host (config-if-Et10-11)# description "Configuration for VM Traffic, VLAN 300, 172.20.1.0/24"
user@host (config-if-Et10-11)# switchport access vlan 300
user@host (config-if-Et10-11)# exit

On the ports connected to MGMT, since we will want to PXE boot, we will need to configure as access ports and make sure spanning-tree port fast is enabled. This keeps the port open long enough for PXE booting to occur. Assuming two ports for the Management Network, ports 20 and 21, we run these commands:

user@host (config)# int eth 20-21
user@host (config-if-Et20-21)# description "Configuration for Management Network, VLAN 100, 10.100.1.0/24"
user@host (config-if-Et20-21)# switchport access vlan 100
user@host (config-if-Et20-21)# spanning-tree portfast
user@host (config-if-Et120-21)# exit

Configure The iLO IPs On The Servers¶

Now that we have the iLO network configured on the switch, we have to manually assign IP addresses to the servers for their dedicated iLO ports. You will have to consult your product manual on how to configure the iLO interface. There are different terms to refer to iLO and it largely depends on the vendor:

Configuring Your Zone¶

This step entails the creation of your Occam configuration files. The configuration files use hiera - a key/value lookup tool for configuration data - built to make Puppet better and let you set node-specific data without repeating yourself. Hiera loads the hierarchy from the hiera.yaml config file.

Occam’s hiera lookup table has the following precedence:

Note: The following represent example hiera files. They may be used as a reference. You can name your zone anything.

Secrets¶

Example secrets file: secrets/zone1.yaml

fqdn¶

Example ops node file: ops1.zone1.example.com.yaml

Example ctrl node file: ctrl1.zone1.example.com.yaml

Example monit node file: monit1.zone1.example.com.yaml

Zone¶

Example zone file: zone1.yaml

You can also generate zone file using rake task config:generate

Hostgroups¶

Example host files:

• ops.yaml

• ctrl.yaml

• comp.yaml

• monit.yaml

• occam-node.yaml

Users¶

Example users file: users.yaml

Occam¶

Example occam file: occam.yaml

Checking out Occam repository¶

% git checkout https://github.com/att-innovate/occam.git
% cd occam

Setup RVM and install the necessary gems¶

1. Install rvm if you do not have it:

% curl -sSL https://get.rvm.io | bash

2. Install the right ruby version:

% rvm install ruby-1.8.7

3. Install the necessary gems

% bundle install

Adding local configuration¶

Local configuration directory should contain everything needed to deploy your zones. It’s capable of storing as many zones as you need. Zone to deploy is selected during installation process. It’s the best to keep the configuration as a git module, but you can just unpack a tarball with configuration if you prefer.

Directory structure is as follows:

local/
  hiera/
    fqdns/
    hostgroups/
    secrets/
    users/
    zones/
  ssl/

local/ssl

This directory should contain SSL certificates in PEM format to be used in a zone. A rake task will put each file from this directory to puppet/modules/profile/files/ssl

local/hiera

Directory that reflects part of the occam’s hiera hierarchy. Most important (and the only one required) is a zones directory. There you should put yaml files with zones configuration. Others are optional, depending on your setup. Eg. you can use hiera gpg backend to store sensitive information like passwords. In that case just put gpg encrypted yaml file in secrets directory. The secrets directory takes precedens over zones one. See Configuring Your Zone for reference.

If you have local conf ready, add it as a submodule at local directory:

% git submodule add https://your.repo/config.git local
% git submodule init
% git submodule update

Initialization of Occam itself¶

Initialize occam:

% rake occam:init

Installation and initialization of Occam application¶

Occam applications are configured as a list in the zone file under the key profile::hiera::config::occam_apps.

profile::hiera::config::occam_apps:
  - 'att-innovate/occam-havana-cloud'

% rake apps:init

% rake apps:clean

Prepare Your OPS Node¶

This step also includes installing Ubuntu on the OPS node, configuring the initial administrator user and configuring initial networking.

1. Install Ubuntu OS on the OPS Node. (Using Virtual Console from the iDRAC web GUI).

Note: Only for OPS node, perform the above step on Windows. Open the virtual console, select the virtual mdeia tab and click lauch Virtual Media. This will open a new box, where you can select the folder in the your workstation that has the ubuntu installation iso. Once done, check the “Mapped” box and keep the dialog box open until the Ubuntu install. Now Select the net boot to “Virtual CD/DVD/ISO” and reboot the box.

During installation set OPS Node to these values: (they have to be consistent with the zone file variables)

Hostname: ops1.zone1.example.com (orchestrator+mgmt_domain from the zone file) Network configuration:

eth0 (mgmt_interface from the zone file):
  ip address: 10.100.1.10 (puppet_address from the zone file)
  network: 255.255.255.0 (mgmt_network from the zone file)
  gateway: 10.100.1.1 (mgmt_gateway from the zone file)
  dns server: 8.8.8.8 (external_dns from the zone file)

2. Check connectivity

Check if you can reach gateway and the Internet from ops1:

$ root@ops1:~# ping 10.100.1.1
$ root@ops1:~# ping google.com

3. Deploy Occam on the Ops Node:

% OPSUSERNAME='root' OPSPASSWORD='secretpassword' OC_ENVIRONMENT=testing ZONEFILE=yourzone rake occam:deploy_initial\[10.100.1.10\]

Where:

• OPSUSERNAME - username on ops1 node
• OPSPASSWORD - password for OPSUSERNAME
• OC_ENVIRONMENT - name of your puppet environment, usually testing or production
• ZONEFILE - zone configuration file from puppet/hiera/zones directory without .yaml extension. Puppet on ops node uses this information to read configuration. You can have many zones within one Occam project
• 10.100.1.10 - ip address of ops node

This rake task will package and transfer Occam folder to /var/puppet/environments/$OC_ENVIRONMENT/ on ops node and then install and configure all ops services like puppet, hiera, etc.

4. Validate Occam deploy on Ops Node:

$ root@ops1:~# netstat -ntlp

5. Boot Controller and compute nodes.
6. Login to Controller after successful boot and validate nova services:

$ root@ctrl1:~# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth ctrl1                                internal         enabled    :-)   2013-10-20 19:40:28
nova-scheduler   ctrl1                                internal         enabled    :-)   2013-10-20 19:40:28
nova-conductor   ctrl1                                internal         enabled    :-)   2013-10-20 19:40:25
nova-compute     comp1                                nova             enabled    :-)   2013-10-19 22:31:42
nova-cert        ctrl1                                internal         enabled    :-)   2013-10-20 19:40:28